Bioinformatics and Genomics File Formats Reference

This reference covers file formats used in genomics, transcriptomics, sequence analysis, and related bioinformatics applications.

Sequence Data Formats

.fasta / .fa / .fna - FASTA Format

Description: Text-based format for nucleotide or protein sequences Typical Data: DNA, RNA, or protein sequences with headers Use Cases: Sequence storage, BLAST searches, alignments Python Libraries:

• Biopython: SeqIO.parse('file.fasta', 'fasta')
• pyfaidx: Fast indexed FASTA access
• screed: Fast sequence parsing EDA Approach:
• Sequence count and length distribution
• GC content analysis
• N content (ambiguous bases)
• Sequence ID parsing
• Duplicate detection
• Quality metrics for assemblies (N50, L50)

.fastq / .fq - FASTQ Format

Description: Sequence data with base quality scores Typical Data: Raw sequencing reads with Phred quality scores Use Cases: NGS data, quality control, read mapping Python Libraries:

• Biopython: SeqIO.parse('file.fastq', 'fastq')
• pysam: Fast FASTQ/BAM operations
• HTSeq: Sequencing data analysis EDA Approach:
• Read count and length distribution
• Quality score distribution (per-base, per-read)
• GC content and bias
• Duplicate rate estimation
• Adapter contamination detection
• k-mer frequency analysis
• Encoding format validation (Phred33/64)

.sam - Sequence Alignment/Map

Description: Tab-delimited text format for alignments Typical Data: Aligned sequencing reads with mapping quality Use Cases: Read alignment storage, variant calling Python Libraries:

• pysam: pysam.AlignmentFile('file.sam', 'r')
• HTSeq: HTSeq.SAM_Reader('file.sam') EDA Approach:
• Mapping rate and quality distribution
• Coverage analysis
• Insert size distribution (paired-end)
• Alignment flags distribution
• CIGAR string patterns
• Mismatch and indel rates
• Duplicate and supplementary alignment counts

.bam - Binary Alignment/Map

Description: Compressed binary version of SAM Typical Data: Aligned reads in compressed format Use Cases: Efficient storage and processing of alignments Python Libraries:

• pysam: Full BAM support with indexing
• bamnostic: Pure Python BAM reader EDA Approach:
• Same as SAM plus:
• Compression ratio analysis
• Index file (.bai) validation
• Chromosome-wise statistics
• Strand bias detection
• Read group analysis

.cram - CRAM Format

Description: Highly compressed alignment format Typical Data: Reference-compressed aligned reads Use Cases: Long-term storage, space-efficient archives Python Libraries:

• pysam: CRAM support (requires reference)
• Reference genome must be accessible EDA Approach:
• Compression efficiency vs BAM
• Reference dependency validation
• Lossy vs lossless compression assessment
• Decompression performance
• Similar alignment metrics as BAM

.bed - Browser Extensible Data

Description: Tab-delimited format for genomic features Typical Data: Genomic intervals (chr, start, end) with annotations Use Cases: Peak calling, variant annotation, genome browsing Python Libraries:

• pybedtools: pybedtools.BedTool('file.bed')
• pyranges: pyranges.read_bed('file.bed')
• pandas: Simple BED reading EDA Approach:
• Feature count and size distribution
• Chromosome distribution
• Strand bias
• Score distribution (if present)
• Overlap and proximity analysis
• Coverage statistics
• Gap analysis between features

.bedGraph - BED with Graph Data

Description: BED format with per-base signal values Typical Data: Continuous-valued genomic data (coverage, signals) Use Cases: Coverage tracks, ChIP-seq signals, methylation Python Libraries:

• pyBigWig: Can convert to bigWig
• pybedtools: BedGraph operations EDA Approach:
• Signal distribution statistics
• Genome coverage percentage
• Signal dynamics (peaks, valleys)
• Chromosome-wise signal patterns
• Quantile analysis
• Zero-coverage regions

.bigWig / .bw - Binary BigWig

Description: Indexed binary format for genome-wide signal data Typical Data: Continuous genomic signals (compressed and indexed) Use Cases: Efficient genome browser tracks, large-scale data Python Libraries:

• pyBigWig: pyBigWig.open('file.bw')
• pybbi: BigWig/BigBed interface EDA Approach:
• Signal statistics extraction
• Zoom level analysis
• Regional signal extraction
• Efficient genome-wide summaries
• Compression efficiency
• Index structure analysis

.bigBed / .bb - Binary BigBed

Description: Indexed binary BED format Typical Data: Genomic features (compressed and indexed) Use Cases: Large feature sets, genome browsers Python Libraries:

• pybbi: BigBed reading
• pybigtools: Modern BigBed interface EDA Approach:
• Feature density analysis
• Efficient interval queries
• Zoom level validation
• Index performance metrics
• Feature size statistics

.gff / .gff3 - General Feature Format

Description: Tab-delimited format for genomic annotations Typical Data: Gene models, transcripts, exons, regulatory elements Use Cases: Genome annotation, gene prediction Python Libraries:

• BCBio.GFF: Biopython GFF module
• gffutils: gffutils.create_db('file.gff3')
• pyranges: GFF support EDA Approach:
• Feature type distribution (gene, exon, CDS, etc.)
• Gene structure validation
• Strand balance
• Hierarchical relationship validation
• Phase validation for CDS
• Attribute completeness
• Gene model statistics (introns, exons per gene)

.gtf - Gene Transfer Format

Description: GFF2-based format for gene annotations Typical Data: Gene and transcript annotations Use Cases: RNA-seq analysis, gene quantification Python Libraries:

• pyranges: pyranges.read_gtf('file.gtf')
• gffutils: GTF database creation
• HTSeq: GTF reading for counts EDA Approach:
• Transcript isoform analysis
• Gene structure completeness
• Exon number distribution
• Transcript length distribution
• TSS and TES analysis
• Biotype distribution
• Overlapping gene detection

.vcf - Variant Call Format

Description: Text format for genetic variants Typical Data: SNPs, indels, structural variants with annotations Use Cases: Variant calling, population genetics, GWAS Python Libraries:

• pysam: pysam.VariantFile('file.vcf')
• cyvcf2: Fast VCF parsing
• PyVCF: Older but comprehensive EDA Approach:
• Variant count by type (SNP, indel, SV)
• Quality score distribution
• Allele frequency spectrum
• Transition/transversion ratio
• Heterozygosity rates
• Missing genotype analysis
• Hardy-Weinberg equilibrium
• Annotation completeness (if annotated)

.bcf - Binary VCF

Description: Compressed binary variant format Typical Data: Same as VCF but binary Use Cases: Efficient variant storage and processing Python Libraries:

• pysam: Full BCF support
• cyvcf2: Optimized BCF reading EDA Approach:
• Same as VCF plus:
• Compression efficiency
• Indexing validation
• Read performance metrics

.gvcf - Genomic VCF

Description: VCF with reference confidence blocks Typical Data: All positions (variant and non-variant) Use Cases: Joint genotyping workflows, GATK Python Libraries:

• pysam: GVCF support
• Standard VCF parsers EDA Approach:
• Reference block analysis
• Coverage uniformity
• Variant density
• Genotype quality across genome
• Reference confidence distribution

RNA-Seq and Expression Data

.counts - Gene Count Matrix

Description: Tab-delimited gene expression counts Typical Data: Gene IDs with read counts per sample Use Cases: RNA-seq quantification, differential expression Python Libraries:

• pandas: pd.read_csv('file.counts', sep='\ ')
• scanpy (for single-cell): sc.read_csv() EDA Approach:
• Library size distribution
• Detection rate (genes per sample)
• Zero-inflation analysis
• Count distribution (log scale)
• Outlier sample detection
• Correlation between replicates
• PCA for sample relationships

.tpm / .fpkm - Normalized Expression

Description: Normalized gene expression values Typical Data: TPM (transcripts per million) or FPKM values Use Cases: Cross-sample comparison, visualization Python Libraries:

• pandas: Standard CSV reading
• anndata: For integrated analysis EDA Approach:
• Expression distribution
• Highly expressed gene identification
• Sample clustering
• Batch effect detection
• Coefficient of variation analysis
• Dynamic range assessment

.mtx - Matrix Market Format

Description: Sparse matrix format (common in single-cell) Typical Data: Sparse count matrices (cells × genes) Use Cases: Single-cell RNA-seq, large sparse matrices Python Libraries:

• scipy.io: scipy.io.mmread('file.mtx')
• scanpy: sc.read_mtx('file.mtx') EDA Approach:
• Sparsity analysis
• Cell and gene filtering thresholds
• Doublet detection metrics
• Mitochondrial fraction
• UMI count distribution
• Gene detection per cell

.h5ad - Anndata Format

Description: HDF5-based annotated data matrix Typical Data: Expression matrix with metadata (cells, genes) Use Cases: Single-cell RNA-seq analysis with Scanpy Python Libraries:

• scanpy: sc.read_h5ad('file.h5ad')
• anndata: Direct AnnData manipulation EDA Approach:
• Cell and gene counts
• Metadata completeness
• Layer availability (raw, normalized)
• Embedding presence (PCA, UMAP)
• QC metrics distribution
• Batch information
• Cell type annotation coverage

.loom - Loom Format

Description: HDF5-based format for omics data Typical Data: Expression matrices with metadata Use Cases: Single-cell data, RNA velocity analysis Python Libraries:

• loompy: loompy.connect('file.loom')
• scanpy: Can import loom files EDA Approach:
• Layer analysis (spliced, unspliced)
• Row and column attribute exploration
• Graph connectivity analysis
• Cluster assignments
• Velocity-specific metrics

.rds - R Data Serialization

Description: R object storage (often Seurat objects) Typical Data: R analysis results, especially single-cell Use Cases: R-Python data exchange Python Libraries:

• pyreadr: pyreadr.read_r('file.rds')
• rpy2: For full R integration
• Conversion tools to AnnData EDA Approach:
• Object type identification
• Data structure exploration
• Metadata extraction
• Conversion validation

Alignment and Assembly Formats

.maf - Multiple Alignment Format

Description: Text format for multiple sequence alignments Typical Data: Genome-wide or local multiple alignments Use Cases: Comparative genomics, conservation analysis Python Libraries:

• Biopython: AlignIO.parse('file.maf', 'maf')
• bx-python: MAF-specific tools EDA Approach:
• Alignment block statistics
• Species coverage
• Gap analysis
• Conservation scoring
• Alignment quality metrics
• Block length distribution

.axt - Pairwise Alignment Format

Description: Pairwise alignment format (UCSC) Typical Data: Pairwise genomic alignments Use Cases: Genome comparison, synteny analysis Python Libraries:

• Custom parsers (simple format)
• bx-python: AXT support EDA Approach:
• Alignment score distribution
• Identity percentage
• Syntenic block identification
• Gap size analysis
• Coverage statistics

.chain - Chain Alignment Format

Description: Genome coordinate mapping chains Typical Data: Coordinate transformations between genome builds Use Cases: Liftover, coordinate conversion Python Libraries:

• pyliftover: Chain file usage
• Custom parsers for chain format EDA Approach:
• Chain score distribution
• Coverage of source genome
• Gap analysis
• Inversion detection
• Mapping quality assessment

.psl - Pattern Space Layout

Description: BLAT/BLAST alignment format Typical Data: Alignment results from BLAT Use Cases: Transcript mapping, similarity searches Python Libraries:

• Custom parsers (tab-delimited)
• pybedtools: Can handle PSL EDA Approach:
• Match percentage distribution
• Gap statistics
• Query coverage
• Multiple mapping analysis
• Alignment quality metrics

Genome Assembly and Annotation

.agp - Assembly Golden Path

Description: Assembly structure description Typical Data: Scaffold composition, gap information Use Cases: Genome assembly representation Python Libraries:

• Custom parsers (simple tab-delimited)
• Assembly analysis tools EDA Approach:
• Scaffold statistics (N50, L50)
• Gap type and size distribution
• Component length analysis
• Assembly contiguity metrics
• Unplaced contig analysis

.scaffolds / .contigs - Assembly Sequences

Description: Assembled sequences (usually FASTA) Typical Data: Assembled genomic sequences Use Cases: Genome assembly output Python Libraries:

• Same as FASTA format
• Assembly-specific tools (QUAST) EDA Approach:
• Assembly statistics (N50, N90, etc.)
• Length distribution
• Coverage analysis
• Gap (N) content
• Duplication assessment
• BUSCO completeness (if annotations available)

.2bit - Compressed Genome Format

Description: UCSC compact genome format Typical Data: Reference genomes (highly compressed) Use Cases: Efficient genome storage and access Python Libraries:

• py2bit: py2bit.open('file.2bit')
• twobitreader: Alternative reader EDA Approach:
• Compression efficiency
• Random access performance
• Sequence extraction validation
• Masked region analysis
• N content and distribution

.sizes - Chromosome Sizes

Description: Simple format with chromosome lengths Typical Data: Tab-delimited chromosome names and sizes Use Cases: Genome browsers, coordinate validation Python Libraries:

• Simple file reading with pandas
• Built into many genomic tools EDA Approach:
• Genome size calculation
• Chromosome count
• Size distribution
• Karyotype validation
• Completeness check against reference

Phylogenetics and Evolution

.nwk / .newick - Newick Tree Format

Description: Parenthetical tree representation Typical Data: Phylogenetic trees with branch lengths Use Cases: Evolutionary analysis, tree visualization Python Libraries:

• Biopython: Phylo.read('file.nwk', 'newick')
• ete3: ete3.Tree('file.nwk')
• dendropy: Phylogenetic computing EDA Approach:
• Tree structure analysis (tips, internal nodes)
• Branch length distribution
• Tree balance metrics
• Ultrametricity check
• Bootstrap support analysis
• Topology validation

.nexus - Nexus Format

Description: Rich format for phylogenetic data Typical Data: Alignments, trees, character matrices Use Cases: Phylogenetic software interchange Python Libraries:

• Biopython: Nexus support
• dendropy: Comprehensive Nexus handling EDA Approach:
• Data block analysis
• Character type distributi