Brief IA
›

Robots: Companies Must Choose Which Data to Keep

Business & Startups·Tom Levy·

Robots: Companies Must Choose Which Data to Keep

Robots: Companies Must Choose Which Data to Keep
Key Takeaways
1An autonomous vehicle can produce up to 5 terabytes per hour.
2No universally accepted retention period exists for robotic data.
3Information surrounding an incident is typically preserved in secure storage.
Why it matters : Deleting data can hinder the complete reconstruction of an AI's state during a contested decision.

Companies using robots are required to adjust data retention periods based on each application, with no standard duration being recognized. An autonomous vehicle can generate up to 5 terabytes of data every hour. Depending on whether it involves analyzing incidents, training, or ensuring individual protection, the required information and the relevant durations differ.

An Incident Requires Preserved Data

After an accident, malfunction, or anomaly, investigating the root cause requires more than just an error log. The inquiry may involve all exchanged content, state changes, events affecting the infrastructure or system, triggers, and logs. This information is typically isolated around the event and then transferred or duplicated in a protected space. Its consultation may occur asynchronously, hours or days later.

The ability to understand a decision also depends on the synchronization of data streams. Robotic reactions rely on the temporal relationships between multiple modalities; therefore, precise timestamps are recommended to correlate causes and effects. From a temporal reference point, an engineer can retrieve associated information. Conversely, the deletion of archives may make it impossible to fully reconstruct the state of the system at the time of a disputed decision.

Compliance particularly aims at this ability to explain behavior or trace the origin of an undesirable reaction. While operators may not have control over the internal design of the machines, they often have significant latitude regarding the information captured, its formats, correlations, and protection. Their management thus encompasses collection, organization, synchronization, and storage.

Each Use Imposes Its Retention Horizon

Learning and compliance are the main reasons for retaining robotic data. However, immediate operation, troubleshooting, training, and monitoring do not require the same precision or speed of access. Operational information supports immediate decisions and system functioning, with very low latency and detailed telemetry. It can also aid alerts, error processing, or service commitment compliance; it generally remains available for several hours, depending on business and regulatory needs.

Troubleshooting aims to quickly correct present symptoms. It relies on state changes, recent events, errors, and traces, with an acceptable latency ranging from a few seconds to 60 minutes. Retention typically extends from a few hours to a few days.

Learning includes simulation, error detection, and behavior adjustment. Without latency requirements, training utilizes large historical datasets comprising feature matrices, annotations, and data divided into subsets. These datasets are archived long-term, with version tracking and management.

Extended monitoring, on the other hand, seeks to identify data or model drifts as well as the emergence of biases. Periodic aggregations are usually sufficient: simplified time series, statistics, and evolution of service commitment compliance. Retention can then last for months or years to meet compliance and capacity planning needs.

La lettre du soirGratuit · 19 h

Le brief IA que les pros lisent chaque soir

Les 7 actus IA du jour, décryptées en 5 min. Gratuit.

Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.

Choisissez votre rythme

Gratuit · Pas de spam · Désabonnement en 1 clic

Proposed Durations Remain Guidelines

In the absence of a common duration for all applications, the choice depends on the nature of the information, the relevant sector, and operational constraints. Business, technical, and compliance functions are called upon to collaboratively establish appropriate periods. For video, audio, and sensory streams used in decision-making, 90 days is a proposed guideline, intended to allow time to examine an incident or conduct corrective training. A legal or operational need may justify extended retention.

For error and maintenance logs typically produced by robots, the recommendation is one to three years. The horizon should align with predictive maintenance cycles and performance evaluations: a machine serviced every two years, for example, provides a reason to retain its logs for that long. In heavily regulated activities, inspection information should follow the regulatory lifespan of parts, with guidelines ranging from five to ten years. ISO 9001, FDA 21 CFR Part 11, IATF 16949, and AS9100 are among the references cited for this traceability.

Faces, voices, or movements captured by certain robots may constitute sensitive or identifiable information. This data falls under regulatory protections, notably the GDPR: its collection and retention must remain linked to a specific purpose, with appropriate security measures. A retention period of 30 to 90 days is proposed as a guideline, not as a general obligation independent of this purpose.

Onboard Storage Limits Available Data

Autonomous vehicles transporting passengers or goods, like robots collaborating with humans, produce significant volumes of data. The streams come from vision devices, microphones, and environmental, force, or torque sensors. Their management exposes companies to risks of confidentiality, security, and expenses.

Their quality also impacts physical safety: defective information can contribute to serious, even fatal, accidents. A camera with an obstructed view or a noisy environment masking a useful signal can disrupt operations. Meanwhile, very frequent streams must be recorded, processed, and utilized within milliseconds. Storage must adhere to these timing constraints.

However, most machines cannot retain everything they produce, especially when their size, weight, or energy autonomy is constrained. Depending on the application, they may transfer part or all of the data to a data center or the cloud. A circular buffer is another possibility: old, no longer useful elements are replaced in the order they were received. Compression and reduction can also increase available capacity, particularly by decreasing the size of raw video images.

Remote storage does not eliminate economic constraints: large volumes can potentially incur recurring costs, along with transfer fees. Traditional raw file storage may also be ill-suited for continuous high-throughput streams. Formats designed for robotic capture are proposed, including ROS bags and multimodal message capture formats.

Selection Precedes Storage Choice

Before choosing caching or archiving, the recommended approach is to examine the produced data, determine which to keep, and set their availability duration. The intended uses include continuous training, behavior adjustments, compliance, and potential commercialization. Monetization remains a possibility, not an assured revenue.

Eliminating unnecessary information saves space, but data not retained is no longer available after processing. Therefore, selection must integrate the needs for subsequent analysis as well as applicable obligations. Long-term management then includes the backup, protection, and deletion of retained information.

Brief IA — L'actualité IA en français

L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.