What is Data? The Definition, Forms & Management

What are Data and Data Forms?

Data Definition: Data is the result of facts or observations, a logical induction of objective phenomena, and the unprocessed raw material used to represent objective entities. It can be continuous values, such as sound or images, referred to as analog data; or discrete values, such as symbols or text, referred to as digital data. In computer science, data is the collective term for all symbols that can be input into a computer and processed by computer programs.

It refers to the general designation for numbers, letters, symbols, and analog quantities with specific meaning that are input into electronic computers for processing.

It possesses accuracy and completeness. Driven by the digital revolution, data is growing at an unprecedented pace, becoming an indispensable foundation for modern socioeconomic activities.

Data Forms: Data comes in various forms and can be broadly categorized into the following types based on its structure and processing methods:

• Structured Data: This type of data is typically stored in relational databases and possesses a fixed format and structure, such as fields for name, age, address, etc. Structured data is easy to manage and analyze, making it the preferred choice for many businesses and organizations conducting data mining and decision support.
• Unstructured Data: In contrast to structured data, unstructured data lacks a fixed format or structure, encompassing text, images, audio, video, and similar formats. This type of data dominates fields like social media and e-commerce, characterized by its vast volume and flexible forms. With the continuous advancement of big data technologies, the capabilities for mining and analyzing unstructured data are also steadily improving.
• Semi-structured data: Positioned between structured and unstructured data, semi-structured data possesses some structure but not as rigidly defined as structured data. Common examples include documents in formats like XML and JSON. This type of data finds extensive application in web development, data exchange, and related fields.

How to Manage Data in Enterprise Environment

How to organize and manage data to leverage and maximize data, especially for enterprise. Read this part to know more details.

1. Data Value Realization: Data Collision, Integration and Sharing, Circulation

The value of data lies not only in its volume but also in the extent to which it is utilized and mined. In the digital age, collision, integration, sharing, and circulation of diverse data can unleash tremendous value, driving social progress and economic development.

• Diverse Data Integration: The collision of multidimensional data: This refers to the interweaving of data from different sources and types, forming new data combinations and associations. Such collisions can spark new insights and discoveries, providing strong support for business innovation and decision-making optimization. For example, in the financial industry, by integrating multi-source data such as customer transaction records, credit history, and social media data, it is possible to more accurately assess a customer’s credit status and risk appetite, thereby offering a basis for credit approval and risk management decisions.

• Data integration and sharing: Data integration and sharing refer to the consolidation and exchange of data across different domains and systems, breaking down data silos to achieve interoperability. This approach enables the construction of more comprehensive and accurate data views, providing robust support for business collaboration and decision-making.

For instance, in smart city development, integrating data from multiple sectors such as transportation, environment, and healthcare allows for real-time monitoring and early warning of urban operations, offering scientific foundations for city management and public services.

• The circulation of data: The circulation of data is a crucial link in realizing its value. Only by allowing data to flow at the right time and in the right way can its value be fully utilized. The circulation of data facilitates the transmission and sharing of information, driving business collaboration and innovation.

For instance, in the field of e-commerce, data circulation enables real-time updates and sharing of product information, providing consumers with a more personalized shopping experience. At the same time, merchants can analyze consumers’ shopping behaviors and preferences to optimize product recommendations and marketing strategies, thereby increasing sales and customer satisfaction.

2. Data Asset Incorporation into financial statements/Data Asset Accounting

This is a process through which an organization registers, classifies, evaluates, and manages various data assets, ultimately incorporating them into financial statements. This process generally involves three steps: Data Resourcing, Resource Productization, and Product Assetization.

• Data Resourcing: This is the process by which an enterprise transforms raw data into resources with potential value. It involves activities such as data collection, compliance assessment, and data governance. Raw data may originate from the organization’s daily business operations, collaborations with public service entities, or data trading markets. Enterprises must ensure compliance in terms of data sources, content, processing, management, and operations. By investing significant human, organizational, technological, and systemic resources, they convert raw data into valuable data resources.
• Data Resource Productization: This refers to the process of transforming valuable data resources into practical and applicable data products. Enterprises define specific application scenarios for the data and deeply integrate valuable data content with service endpoints or algorithms to ensure that the data can be utilized in an intuitive and efficient manner. Data products may be used internally or brought to market for commercial transactions.
• Data Product Assetization: During the management of data resources, once the utility value of data resources is identified and further refined into data products, these products serve as tangible carriers for asset recognition. Upon meeting specific criteria, they are incorporated into financial statements. These criteria include the enterprise’s legal ownership or control over the data product, the expectation that the data product will generate economic benefits for the enterprise (i.e., monetizability), and the ability to reliably measure the cost of the data product (i.e., quantifiability).

3. Replication and circulation transform data into wealth

In the digital era, data has emerged as a new form of wealth. The replication and circulation of data can dismantle data monopolies and barriers, promote data sharing and utilization, and drive business collaboration and innovation.

• Data Replication

Data replication refers to the process of copying data from one location or system to another. Through replicating data, enterprises can achieve data backup and recovery, preventing data loss and corruption. Furthermore, it enables distributed data storage and access, enhancing data availability and retrieval speed. In the digital age, data replication has become a universal necessity and practice.

• Data Circulation

Data circulation refers to the flow and sharing of data across different systems and platforms. Data circulation facilitates interconnectivity and synergy between disparate systems. Concurrently, it promotes data sharing and utilization, thereby driving business collaboration and innovation. In the digital age, data circulation has become a significant business model and a means of profit generation. For instance, in data exchange markets, the buying and selling of data enables value realization and appreciation of the data. On data sharing platforms, collaboration and sharing facilitate business synergy and innovative development.

• Data as Wealth

With the advent of the digital era and the continuous advancement of data technology, data has become a new form of wealth. The value of data lies not only in its volume and quality but, critically, in the extent to which it is utilized and mined. Through data replication and circulation, the full value of data can be realized, driving business collaboration and innovative growth. Consequently, an increasing number of enterprises and organizations are treating data as a vital asset and resource, actively investing capital and technology into data collection, storage, processing, and analysis. Simultaneously, governments and society are focusing on the value and role of data, advocating for the formulation and implementation of data sharing and open data policies.

How to achieve Data Security

Obviously, data is vital for business and also people. Yet every byte of this valuable information exists within a digital environment rife with threats. On average, companies lose $4.45 million per data breach, and this figure continues to rise annually. This explains why more businesses are discussing data security. 

Data security technologies are the core mechanisms for safeguarding data integrity and confidentiality. They are critical for any organization operating in today’s digital landscape. These technologies fundamentally include: encryption, identity authentication and access control, data masking, backup, and security auditing and monitoring.

Next, we will explain them one by one.

Data Encryption Technology

Encryption is the process of transforming data into an unreadable format (ciphertext) to protect its confidentiality and integrity. It primarily consists of two major types:

Symmetric Encryption: 

• Description: Also known as secret-key or shared-key encryption, it uses the same single key for both encrypting and decrypting data.

• Key Attributes: This method is fast and easy to implement, making it suitable for encrypting large volumes of data efficiently.

• Vulnerability: A significant drawback is the risk associated with key distribution; if the shared key is intercepted during transmission, the security of the encrypted data is compromised.

• Common Algorithms: DES (Data Encryption Standard) and AES (Advanced Encryption Standard).

Asymmetric Encryption

• Description: Unlike symmetric encryption, asymmetric encryption uses a key pair: a public key and a private key. The public key is openly shared and used by anyone to encrypt a message; the private key is kept secret and is the only key capable of decrypting the message.

• Security Advantage: This method offers enhanced security because even if an attacker intercepts the encrypted message, they cannot decrypt it without the corresponding private key. This is essential for secure communication and digital signatures.

• Common Algorithms: RSA and ECC (Elliptic Curve Cryptography).

Identity Authentication and Access Control

Identity authentication and access control technologies are fundamental to the principle of “Least Privilege,” ensuring that only authorized personnel can access specific data resources by verifying user identity and restricting data access permissions.

Identity Authentication (Authentication): This is the process of verifying a user’s claimed identity. Typically achieved via usernames and passwords, biometric technologies (such as fingerprint or facial recognition), or hardware tokens.

Access Control (Authorization): Restrict access to data based on user identity and permissions. Common access control models include Role-Based Access Control (RBAC) and Attribute-Based Access Control (ABAC). These models can assign different access permissions based on a user’s role or attributes.

Data Masking (De-identification)

Data masking technology involves processing sensitive data to conceal or remove private information, thereby protecting data privacy and security. This technique is essential for ensuring that sensitive data is not exposed in non-production environments such as development, testing, and quality assurance.

Common data masking methods include:

• Tokenization/Substitution: Generating new data that conforms to the original data’s coding and validation rules, which is then used to replace the actual sensitive data. This maintains data utility for testing while protecting privacy.

• Data Replacement: Replacing sensitive content with patterned characters or generic strings to destroy the data’s readability and render it meaningless outside the intended context.

• Encryption: Applying encryption algorithms to sensitive data. While the data remains reversible, access is restricted, often used for data that might need to be re-identified later under strict control.

• Data Truncation: Selecting and cutting off a portion of the original data content, such as showing only the last four digits of an account number.

• Data Shuffling/Scrambling: Randomly mixing or disordering the content of sensitive data within a column (e.g., swapping names among records) to obscure the original link between the identifier and the record holder.

Data Backup and Recovery

Data backup and recovery technologies are critical safeguards against data loss or corruption. By implementing regular data backups and establishing comprehensive data recovery mechanisms, organizations can ensure the rapid and reliable restoration of data in the event of loss, damage, or disaster.

Common data backup methods include:

• External Storage Devices: Utilizing physical media such as external hard drives, USB flash drives, or Network Attached Storage (NAS) devices for local, direct data backup.

• Cloud Backup: Uploading data to a cloud storage service provider’s servers, which enables automated backups, off-site storage, and remote accessibility for disaster recovery.

• Network Backup: Employing specialized network backup software to transmit data to remote dedicated servers or data centers, facilitating both automated and incremental backups (backing up only the data that has changed since the last full backup).

Security Auditing and Monitoring

Security auditing and monitoring technologies are essential for maintaining continuous visibility into the security posture of an environment. By monitoring and logging data usage and flow in real time, organizations can detect and address security incidents promptly.

This includes several key technological means:

• Log Analysis: Analyzing system and application logs to identify anomalous behavior or potential security threats, often using Security Information and Event Management (SIEM) solutions.

• Intrusion Detection: Employing Intrusion Detection Systems (IDS) to monitor network traffic and system activity in real time, with the goal of detecting unauthorized access attempts or malicious activities.

• Security Incident Management (SIM): The systematic process of collecting, analyzing, and responding to security events. This ensures a rapid and coordinated response to threats, minimizing potential damage and loss.

The implementation of robust security measures, as detailed above, must be integrated seamlessly within an organization’s overall data lifecycle. This brings us to the next crucial component of modern data strategy: Data Processing and Management.

Data Processing and Management

The data processing and data management stages encompass multiple interconnected aspects, including data acquisition, data cleaning and pre-processing, data storage, data analysis, data visualization, as well as data classification and organization, data encoding, data querying and maintenance, data security and privacy protection, and data governance and standardization. These processes are mutually related and supportive, collectively forming a comprehensive system for effective data processing and management.

Data Processing

Overall, effective Data Processing transforms raw inputs into high-quality, actionable datasets, setting the stage for insightful analysis and secure storage. Here are processes for data processing.

1. Data Acquisition

Data acquisition is the starting point of the data processing lifecycle, involving the collection of raw data from various sources through technologies like sensors, monitoring equipment, and the Internet of Things (IoT).

Key points:

• Defining Goals: Determine the objectives and requirements for data acquisition and select appropriate data sources.

• Designing the Plan: Design the data collection scheme, including collection frequency, sampling rate, and data format.

• Ensuring Quality: Guarantee the accuracy and real-time nature of data collection, preventing data loss or errors.

2. Data Cleaning and Pre-processing

Data cleaning and pre-processing are critical stages in data processing, aiming to remove noise, outliers, and duplicate records from the raw data to enhance overall data quality.

Key Steps:

• Data Deduplication: Identifying and removing redundant data records.

• Missing Value Imputation: Employing appropriate methods (e.g., mean imputation, median imputation, interpolation) to fill in missing values, based on data characteristics and business needs.

• Outlier Detection and Correction: Identifying and handling abnormal values in the data to ensure they do not skew subsequent analysis.

• Data Formatting and Standardization: Converting data into a uniform format and standard for easier subsequent processing and analysis.

3. Data Storage

Data storage involves placing the cleaned and processed data into databases, data lakes, or other storage systems for subsequent access and utilization.

Key Points:

• Technology and Architecture: Selecting appropriate data storage technologies and architecture to ensure data reliability, scalability, and security.

• Storage Scheme Design: Designing the data storage plan, including data partitioning and indexing strategies, to enhance data access efficiency.

• Regular Backup: Performing regular data backups to prevent data loss or corruption.

4. Data Analysis

Data analysis involves deep mining and processing of the stored data to extract valuable information and patterns.

Key Points:

• Statistical Analysis: Applying statistical principles for descriptive and inferential analysis of data.

• Machine Learning (ML): Utilizing ML algorithms for tasks such as data classification, clustering, and prediction.

• Deep Learning (DL): Constructing deep neural network models for more complex analysis and processing of data.

5. Data Visualization

Data visualization involves presenting the analysis results intuitively through charts, graphs, and dashboards, helping users to understand and interpret the data easily.

Key Points:

• Tool Selection: Choosing appropriate visualization tools and technologies, such as Matplotlib or Tableau.

• Design Clarity: Designing clear and concise visualization charts that highlight key information and trends in the data.

• Accuracy and Interactivity: Ensuring the accuracy and real-time nature of visualization charts, supporting user interaction and exploration.

Data Management

Following the effective processing of data, the focus shifts to establishing a robust framework for its longevity and governance. This framework is defined by the critical processes of Data Management.

1. Data Classification and Organization

Classifying and organizing data based on its attributes and business requirements to facilitate subsequent querying and governance.

Key Points:

• Standardization: Establishing unified data classification standards and specifications.

• Structure Design: Designing reasonable data catalogs and indexing structures to improve data retrieval efficiency.

2. Data Encoding

Encoding data to ensure its uniqueness and accuracy.

Key Points:

• Rules and Standards: Formulating unified data encoding rules and standards.

• Processing: Applying encoding processes to data, ensuring its uniqueness and identifiability.

3. Data Query and Maintenance

Providing effective data query functions to support users in quickly obtaining required data, and performing regular data maintenance to ensure data timeliness and accuracy.

Key Points:

• Efficiency: Designing efficient data query interfaces and query statements.

• Upkeep: Regularly updating and maintaining the data, including correction and deletion of records.

4. Data Security and Privacy Protection

Ensuring the confidentiality, integrity, and availability of data, preventing data leakage and unauthorized access.

Key Measures:

• Technological Controls: Employing technical measures such as data encryption, access control, and identity authentication to protect data security.

• Compliance: Adhering to relevant laws, regulations, and industry standards to ensure data processing compliance.

5. Data Governance and Standardization

Storing and managing data according to standard data formats, and defining unified data naming conventions and metadata management rules.

Key Focus Areas:

• Naming and Metadata: Establishing unified data naming conventions and metadata management rules.

• Standardization Processing: Processing data for standardization, ensuring data consistency and comparability.