by Dr. Radut

Bulk Extractor with Record Carving

Bulk Extractor with Record Carving (bulk_extractor-rec) is an enhanced version from original Bulk Extractor.

Currently, it is based on bulk extractor-1.6.0-dev, the version scheme is bulk_extractor-recNUM.

The latest version is bulk_extractor-rec03.

 

Additional scanner plugins

  • evtx - EVTX file and EVTX chunks (with generated file header)
  • ntfsindx - INDX records of $INDEX_ALLOCATION attribute
  • ntfslogfile - RSTR/RCRD records of $LogFile
  • ntfsmft - FILE records of $MFT
  • ntfsusn - USN_RECORD structure records of $UsnJrnl:$J
  • utmp - utmp structure records

 

Screenshot

Command

Bulk Extractor Record Carving Command

GUI

Bulk Extractor Record Carving BEViewer

 

Download

Windows 64bit Binary (command)

bulk_extractor-rec03_x64.zip
(SHA-256: d907a5cb6d8f3309705096b53790ce96e6277373c49e1ac714ea4cd7ae2f19fe)

Windows 64bit Installer (GUI)
bulk_extractor-1.6.0-dev-rec03-windowsinstaller_x64.exe
(SHA-256: a219eeb4184a8cf23f5645b0c79c0cf7eab457c978ad91a362c0f7624c87925a)

On Linux you can build from source code. 
https://github.com/4n6ist/bulk_extractor-rec

Previous release

 

bulk_extractor-rec02_x64.zip
(SHA-256: 62f1c091474a7c1ddcb891e2abfcfcfa7741e3cc4984f6316df94154b533348b)

bulk_extractor-rec01_x64.zip
(SHA-256: 0c3a0ef45f701317572e67b520d284e6210be3514e234da426528f3456d334ad)

Windows 64bit Installer (GUI)
bulk_extractor-1.6.0-dev-rec02-windowsinstaller_x64.exe
(SHA-256: f0151e3c2ccd9b4f232ca2ac06a8f2ed23b67b89fcc4c8f316396c2bc3ceafd1)

bulk_extractor-1.6.0-dev-rec01-windowsinstaller_x64.exe
(SHA-256: b2c1f3916c1df803fbb9b6c699ebf5a70f365bd37b8fe5bb71512e3d157c8064)

License

See original license

 

Background

A carving is a method of recovering. It searches header and/or footer pattern of various data, then extracts these ranges as a file, for instance foremost, scalpel and PhotoRec are file carving tools.

Bulk Extractor also has the similar method which carves and parses various information based on an extensive pattern. It works as file carving tool, but also a small chunk of carving.

Bulk Extractor is designed for developing scanners independently, so I decided to try it. My scanner plugins concept is just carving and carving record.

 

Usage

Command 

Additional scanners are enabled by specifying -e option: 

  1. bulk_extractor -E scanner -o output_directory input_file 
    (runs with only one scanner)
  2. bulk_extractor -x all -e scanner1 -e scanner2 ... -o output_directory input_file
    (runs with multiple scanners)

The following example should work on Windows 10: 

  • To carve out INDX, $LogFile, MFT, USN records from Win7.E01

bulk_extractor -x all -e ntfsindx -e ntfslogfile -e ntfsmft -e ntfsusn -o output Win7.E01

  • In addition to the above, to carve out within hiberfil.sys from Win7.E01

bulk_extractor -x all -e hiberfile -e ntfsindx -e ntfslogfile -e ntfsmft -e ntfsusn -o output Win7.E01

  • To carve out MFT records from Volume E: 

bulk_extractor -E ntfsmft -o output \\.\E:

  • To carve out $LogFile record from ShadowCopy1

bulk_extractor -E ntfslogfile -o output \\.\HarddiskVolumeShadowCopy1
(vssadmin list shadows command lists information of VolumeShadowCopy#)

  • To carve out INDX record from PhysicalDrive0

bulk_extractor -E ntfsindx -o output \\.\PhysicalDrive0

  • To carve out utmp record from Linux.E01

bulk_extractor -E utmp -o output Linux.E01

  • In addition to the above, to carve out within gzipped data from Linux.E01

bulk_extractor -x all -e gzip -e utmp -o output Linux.E01

GUI

Additional scanners are displayed and enabled by "scanners" checkbox.

 

Output

The processing results stores at directory by specifying -o option.

Bulk Extractor Record Carving Output

The same type of records are merged into one file. If some records failed to verify (i.e. fixup inconsistency), these are merged into another one file named "_corruped" suffix. INDX records of $INDEX_ALLOCATION represent a content of a directory, typically, but INDX records of $ObjId and $Secure has the different format so these files named INDX_ObjId-O, INDX_Misx respectively.

An offset and size information stored at "scanner_carved.txt".

 

Features

  • Bulk Extractor is multi-threaded so input data should be processed at high speed. An experiment at my computer processing time is over 100MBytes/sec at all times. And furthermore, it achieves 300MBytes/sec with high-spec CPU and storage.
  • Bulk Extractor has some scanners which expand the data recursively. For instance, hiberfile scanner supports expanding of hibernation file named hiberfil.sys at Windows 7, or gzip scanner supports expanding of gzipped files whose format is widely used in Linux. Thereby it is possible to carve out more data.
  • Bulk Extractor allows any type of data. It supports not only disk, volume, and file system but also memory dump.

 

Limitations

  • Additional scanner plugins have a function of just carving, not parsing. You need another tool to parse the result of carving data. 
  • Bulk Extractor doesn't recognize input data as file system or some type of structured data, for that reason the result of carving data does not always compose of the records of the correct order.
  • There is more than one file system in input disk image, one file of the result of carving data will contain all records of each file system.

 

History

2018/10/17

bulk_extractor-rec03 released. Added evtx scanner.

2018/01/21

bulk_extractor-rec02 released. Improved hiberfile and ntfsusn scanner.

2017/12/12

bulk_extractor-rec01 released.