File Formats Tutorial

DoCTER requires input data to be in csv (comma separated values) files, one of the most widely used non-proprietary formats for tabular data. DoCTER output files are also generated in csv format. Spreadsheet data can easily be saved as csv files; similarly, csv files can be readily viewed, modified and saved in standard spreadsheet software.

DoCTER assumes the first row of the input file is reserved for column headings; however, users are required to specify column numbers rather than column headings when interacting with DoCTER (for example, if the second column contains the text of the documents, users must specify the number 2 when interacting with the tool).

A typical DoCTER input file can be visualized as a table with multiple columns:

• One of the columns may be a unique identifier for the document
• Another column should contain the corresponding document text
• Other columns may contain labels or annotations
• Further columns may contain assorted metadata which may not be relevant to the analysis but will be simply passed through.

Additional input data format requirements are available in the tutorials for each of the DoCTER functions or may be surmised from examining the dummy input data files available for each function.

Many databases of academic literature (e.g. PubMed) will readily output document search results (e.g. titles and abstracts retrieved from the literature based on keyword searches) in csv format which may be directly run through DoCTER.

Topic Extraction (Clustering)

Clusters a set of documents into a user-specified number of bins. For each bin, identifies the defining topics/keywords.

Supervised Clustering

Classifies a set of documents as relevant or non-relevant to a topic. Assigns discrete (1-6) priority scores to each document.

Machine Learning

Classifies a set of documents as relevant or non-relevant to a topic. Assesses the probability of a document being relevant to the topic of interest.