January 27, 2015

IDEA Dictionary: What is a Flat File?

When dealing with data, and all related terminology, you may feel like you’re trying to speak a foreign language. In the interest of keeping things clear and simple, we will introduce commonly-used terms in each newsletter that will help you navigate the data shorthand and learn the lingo. This week, we bring you: flat file.

A flat file is one of the file formats manufacturers can use to load data into the Industry Data Warehouse (IDW). With this format, records are stored in a simple table with no internal hierarchy or relational structure found in an XML or EDI file. Data in flat files are displayed and communicated to the receiving system in a defined, positional layout.

A flat file can either be created as a pipe-delimited text file or an Excel spreadsheet. In 2014, IDEA gave IDW users the ability to upload data directly to the IDW via Excel spreadsheets, eliminating the need to use a text editor. This has resulted in a quicker, more efficient data upload process. 

In the image above, the visual difference between a hierarchical XML file and the table layout of the flat file is evident. They both contain the same data, but convey it in very different structures. An XML file, like EDI files, is automated and comes with its own set of IDEA standards that apply when loading data into the IDW. A flat file, on the other hand, is a simpler format more like a table or list of related records. Within a flat file, the system receiving the file will expect a particular type of information to be located in a specific position in the table. For instance, the system will reject the file, or particular view, if information is placed in the incorrect field or does not follow the correct formatting (I.e. sending alpha characters in a field formatted as numeric).

Flat files can be translated to EDI files through VAN services, such as IDEA’s Industry Data Exchange (IDX).

IDEA-provided flat files have recently been updated to accommodate the 43 critical fields in the Data Certification Program. IDEA is phasing out the EZView Flat File and replacing it with a new set of flat files, each comprised of “like” fields, such as basic product information, packaging, pricing, etc. These flat files are now available to all manufacturer customers through the IDW, and can be used to upload new items, as well as extract existing items to determine missing critical fields.

The IDW also allows greater flexibility for distributors when pulling authorized data from trading partners. Distributors are able to create custom flat files for extracting data using only those fields containing the product information they wish to receive from manufacturers to conduct business.

If you have any questions regarding the meaning of “flat file,” or any other IDW terms, please contact your Data Management Specialist. Requests for future definitions can be sent to communications@idea4industry.com.