NSRLJP - complement to NSRL(http://www.nsrl.nist.gov/) hash library with Japanese edition and software.
Download
NSRLJP_202408.7z (SHA-256: e4baca8bedfffb59098609a4267135dd2242e430e51bc9f757bc2351a80f01b5)
(Hashes: 5,268,826 / Filesize: 319,470,599 bytes)
License
You may use the DATASET freely for personal or commercial and NO WARRANTIES.
Motivation
National Software Reference Library (NSRL) provides Reference Data Set(RDS), which is a collection of digital signatures of known, traceable software applications. NSRLJP is a collection of a dataset which is widely used in Japan. It also includes Japanese edition of Microsoft Windows. NSRLJP is compliant with NSRL RDS data format so you can import NSRLJP into your favorite tools as well.
Details of the data format, please refer to NSRL RDS.
(Updated on 2024-08-29)
In 2023, they have released RDSv3 which is described in detail as following document.
https://s3.amazonaws.com/rds.nsrl.nist.gov/RDS/RDSv3_Docs/RDSv3.pdf
For hash analysis purpose, the previous RDSv2 format is effective. NSRLJP still be kept to RDSv2 format.
A script to generate a result of RDSv2 format is available at NSRLJP Script.
Catalog (NSRLJP_202408)
| No. | Name | Count | Comment (version, etc.) |
| 1 | Windows XP x64 | 10820 | |
| 2 | Windows XP x86 | 16080 | SP3 |
| 3 | Windows 2003 R2 x64 | 16593 | SP2 |
| 4 | Windows 2003 R2 x86 | 14562 | SP1 |
| 5 | Windows Vista x64 | 22242 | SP2 |
| 6 | Windows Vista x86 | 15437 | SP2 |
| 7 | Windows 7 x64 | 40685 | SP1 |
| 8 | Windows 7 x86 | 15685 | SP1 |
| 9 | Windows 2008 x64 | 41249 | SP2 |
| 10 | Windows 2008 R2 x64 | 9334 | SP1 |
| 11 | Windows 8 x64 | 16145 | |
| 12 | Windows 8 x86 | 13000 | |
| 13 | Windows 8.1 x64 | 22199 | Update |
| 14 | Windows 8.1 x86 | 19690 | Update |
| 15 | Windows 2012 x64 | 45842 | |
| 16 | Windows 2012 R2 x64 | 56929 | Update |
| 17 | Windows 2016 | 64059 | |
| 18 | Windows 10 x64 | 261160 | 1511, 1607, 1703, 1709, 1803, 1809, 1903, 1909, 2004, 20H2, 21H2, 22H2 |
| 19 | Windows 10 x86 | 429425 | 1511, 1607, 1703, 1709, 1803, 1809, 1903, 1909, 2004, 20H2, 21H2, 22H2 |
| 20 | Windows 2019 | 47621 | Update (2019-03) |
|
21 |
Windows 11 |
113310 |
21H2, 23H2 |
|
22 |
Windows Server 2022 |
56074 |
Update (2022/03) |
| 23 | Office XP | 4468 | SP3 |
| 24 | Office 2003 | 2210 | SP1 |
| 25 | Office 2007 | 2736 | SP2 |
| 26 | Office 2010 | 13680 | SP2 |
| 27 | Office 2013 | 11757 | SP1 |
| 28 | Office 2019 | 4034 | |
| 29 | .NET Framework 2.x | 491 | v2.0 |
| 30 | .NET Framework 3.x | 872 | v3.0, 3.5 |
| 31 | .NET Framework 4.x | 61429 | v4.0, 4.5-4.5.2, 4.6-4.6.2, 4.7-4.7.2, 4.8 |
| 32 | .NET5.x | 352067 | 5.0.100-5.0.408, 6.0.100-6.0.424, 7.0.100-7.0.410, 8.0.100-8.0.303 |
| 33 | Windows Update 2006 | 8765 | |
| 34 | Windows Update 2007 | 17145 | Except 2007-03 |
| 35 | Windows Update 2008 | 19329 | Except 2008-03 |
| 36 | Windows Update 2009 | 36100 | Except 2009-05 |
| 37 | Windows Update 2010 | 59359 | Except 2010-11 |
| 38 | Windows Update 2011 | 68074 | |
| 39 | Windows Update 2012 | 78872 | Except 2012-09 |
| 40 | Windows Update 2013 | 128706 | |
| 41 | Windows Update 2014 | 425232 | |
| 42 | Windows Update 2015 | 430518 | |
| 43 | Windows Update 2016 | 174308 | 2016-01 ~ 08 |
| 44 | Windows Update 2024 |
395141 |
2024-08 |
| 45 | Google Chrome | 11367 | 37.0.2062.117-118.0.5993.89 |
| 46 | Firefox | 70894 | 0.8-128.0 |
| 47 | Thunderbird | 49973 | 0.4-127.0 |
| 48 | Opera | 5117 | 6.01-12.17 |
| 49 | Adobe Reader | 176609 | 11.0.23,2015(1500630527), DC(2001320064), 2017(1701130180), 2020(2000530636), Latest(2000620034-2300820421) |
| 50 | Explzh | 1281 | v6.06, v7.01-7.78, v8.17.4, v8.30-8.39, v9.33-9.47 |
| 51 | Lhaz | 153 | v1.36, v.2.1.3, v2.2.4, v2.4.0, v.2.5.1, v3.3.0, v3.4.0, v3.5.1 |
| 52 | Forefront Client Security | 585 | |
| 53 | Hidemaru Editor | 2096 | 4.19-9.35 |
| 54 | Hidemaru Mail | 367 | 6.01-7.32 |
| 55 | Sakura Editor | 324 | 1.6.1.0-1.6.6.0, 2.0.4.0-2.2.0.1 |
| 56 | Terapad | 54 | 1.00-1.2.9 |
| 57 | “Lhaca” | 10 | 0.76, 0.97, 1.24 |
| 58 | “Lhaplus” | 26 | 1.71-1.74 |
| 59 | WSUS Offline | 1168094 | 2019-05-06, 2021-04-09 |
| Total | 5268826 |
The dataset is deduplicated based on MD5 and SHA-1. It means files have the same hash at various categories, only one record is registered into one of these categories.
Usage
We confirmed that the following tools support NSRLJP:
- X-ways Forensics
- Autopsy
- Magnet AXIOM
It's probably OSForensics, FTK, and md5deep also support.
Case Example
The following table shows the results that hashes with Windows OS are collated with NSRL, NSRLJP and both.
(Updated on 2024-08-29: Applied NSRL 2024.03.1 RDSv2, NSRLJP_202408)
| OS | Total number of files | (1) NSRL | (2) NSRLJP | (3) NSRL+NSRLJP |
| Windows 11 23H2 [22631.2428] | 173630 | 47751 | 61550 | 75825 |
| Windows 10 22H2 [19045.2006] | 106595 | 68220 | 62482 | 72964 |
| Windows Server 2022 [20348.587] | 101948 | 49126 | 71408 | 71583 |
These OS are a clean install, we understand that NSRLJP includes most of hashes. In practical use, we should NSRL as well because it includes a wide variety of applications. Hash analysis is a traditional approach, but it will become increasingly important because the number of files in storages has increased year by year.
History
2024/08/20 (NSRLJP_202408 - 5,268,826 hash / 319,470,599 bytes)
Addition to Windows 10 (21H2, 22H2), 11 (21H2, 23H2), Windows 2022 (Update 2022/03).
2021/04/24 (NSRLJP_202104 - 4,142,667 hash / 253,337,631 bytes)
Addition to Windows 10 (1909, 2004, 20H2), Windows 2019 (Update 2019/03), and Google Chrome.
2019/05/18 (NSRLJP_201905 - 3,718,659 hash / 228,100,979 bytes)
Addition to Windows 10 (1803, 1809, 1903) and Windows 2019.
2018/02/12 (NSRLJP_201802 - 2,993,931 hash / 180,917,699 bytes)
Addition to Windows 10 (1703, 1709) and Windows 2016, WSUS Offline Update for Windows Update.
2016/09/20 (NSRLJP_201609 - 2,309,928 hash / 158,073,246 bytes)
Addition to Windows 10 (1511, 1607), .NET Framework 4.6.1, 4.6.2 and Windows Update 2016.
2015/08/14 (NSRLJP_201508_rev2 - 1,659,348 hash / 114,136,669 bytes)
Excluded invalid record.
2015/08/11 (NSRLJP_201508)
Addition to Windows 10, .NET Framework 4.5.2, 4.6, Lhaca, Lhaplus and Windows Update 2015.
2014/08/10 (NSRLJP_201408 - 1,096,364 hash / 75,095,909 bytes)
Addition to Windows 8.1, 2012R2 Update, Office 2013 SP1, Office 2010 SP2, .NET Framework 3.5, 4.0, 4.5.1 and Windows Update 2014
2014/01/22 (NSRLJP_201401 - 532,923 hash / 38,245,480 bytes)
Addition to Windows 8.1, Windows 2008 R2 and Windows 2012.
2013/08/03 (NSRLJP_201308 - 401,211 hash / 28,657,266 bytes)
Addition to Windows Update, Adobe and Firefox.
2013/01/27 (NSRLJP_201301 - 284,419 hash / 20,673,998 bytes)
Addition to Windows 8/2012, Office XP/2003/2007/2010/2013 and .NET 2.0/3.0/4.5.