Converting JSON Files to Hepfiles

As a side note, since there are many different so-called legal JSON file formats, it would be nearly impossible to generate a completely robust JSON file to hepfile converter. So, we have attempted to make it as easy as possible with the existing software. This tutorial works with a specific JSON file format that is especially easy to convert to a hepfile but it should be possible to do it with others as well!

import hepfile as hf
import json

Say we have the following json text of events with the same sample data from the Writing hepfiles from dictionaries tutorial.

{
'event1': {
    'jet': {
        'px': [1,2,3],
        'py': [1,2,3]
     },
    'muons': {
        'px': [1,2,3],
        'py': [1,2,3]
     },
    'nParticles': 3
    },
'event2': {
    'jet': {
        'px': [3,4,6,7],
        'py': [3,4,6,7]
     },
    'muons': {
        'px': [3,4,6,7],
        'py': [3,4,6,7],
        },
    'nParticles': 4
    }
}

In python, we can write this to a json file using the following code:

# define a file path
jsonfile = 'json-output.json'

# define this json text as a dictionary
events = {
'event1': {
    'jet': {
        'px': [1,2,3],
        'py': [1,2,3]
     },
    'muons': {
        'px': [1,2,3],
        'py': [1,2,3]
     },
    'nParticles': 3
    },
'event2': {
    'jet': {
        'px': [3,4,6,7],
        'py': [3,4,6,7]
     },
    'muons': {
        'px': [3,4,6,7],
        'py': [3,4,6,7],
        },
    'nParticles': 4
    }
}

# write it out to a json file
with open(jsonfile, 'w') as f:
    json.dump(events, f)

Once we have created our json file, we are at the step that most of you will be at when trying to convert a JSON file to a hepfile.

The first step is to read in the json file

with open(jsonfile, 'r') as f:
    jsondata = json.load(f)

print(type(jsondata))
print(jsondata)

<class 'dict'>
{'event1': {'jet': {'px': [1, 2, 3], 'py': [1, 2, 3]}, 'muons': {'px': [1, 2, 3], 'py': [1, 2, 3]}, 'nParticles': 3}, 'event2': {'jet': {'px': [3, 4, 6, 7], 'py': [3, 4, 6, 7]}, 'muons': {'px': [3, 4, 6, 7], 'py': [3, 4, 6, 7]}, 'nParticles': 4}}

As you can see, this is the same data as the writing hepfiles from dictionaries tutorial but stored in a dictionary instead of a list. So, the next step is to convert this to a list of dictionaries to write to a hepfile.

dictlist = []
for key in jsondata.keys():
    dictlist.append(jsondata[key])

print(dictlist)

[{'jet': {'px': [1, 2, 3], 'py': [1, 2, 3]}, 'muons': {'px': [1, 2, 3], 'py': [1, 2, 3]}, 'nParticles': 3}, {'jet': {'px': [3, 4, 6, 7], 'py': [3, 4, 6, 7]}, 'muons': {'px': [3, 4, 6, 7], 'py': [3, 4, 6, 7]}, 'nParticles': 4}]

So, now we have it in a form that makes it easy to use the existing hepfile software. All we need to do from here is call the hepfile.dict_tools.dict_to_hepfile function.

out_filename = 'output_from_json.h5'
data = hf.dict_tools.dictlike_to_hepfile(dictlist, out_filename)
data

{'_GROUPS_': {'_SINGLETONS_GROUP_': ['COUNTER', 'nParticles'],
  'jet': ['njet', 'px', 'py'],
  'muons': ['nmuons', 'px', 'py']},
 '_MAP_DATASETS_TO_COUNTERS_': {'_SINGLETONS_GROUP_': '_SINGLETONS_GROUP_/COUNTER',
  'jet': 'jet/njet',
  'jet/px': 'jet/njet',
  'jet/py': 'jet/njet',
  'muons': 'muons/nmuons',
  'muons/px': 'muons/nmuons',
  'muons/py': 'muons/nmuons',
  'nParticles': '_SINGLETONS_GROUP_/COUNTER'},
 '_LIST_OF_COUNTERS_': ['_SINGLETONS_GROUP_/COUNTER',
  'jet/njet',
  'muons/nmuons'],
 '_SINGLETONS_GROUP_/COUNTER': [1, 1],
 '_MAP_DATASETS_TO_DATA_TYPES_': {'_SINGLETONS_GROUP_/COUNTER': int,
  'jet/njet': int,
  'jet/px': numpy.int64,
  'jet/py': numpy.int64,
  'muons/nmuons': int,
  'muons/px': numpy.int64,
  'muons/py': numpy.int64,
  'nParticles': numpy.int64},
 '_META_': {},
 'jet/njet': [3, 4],
 'jet/px': [1, 2, 3, 3, 4, 6, 7],
 'jet/py': [1, 2, 3, 3, 4, 6, 7],
 'muons/nmuons': [3, 4],
 'muons/px': [1, 2, 3, 3, 4, 6, 7],
 'muons/py': [1, 2, 3, 3, 4, 6, 7],
 'nParticles': [3, 4]}

So, while it is difficult to create software to convert every possible JSON file format out there to a hepfile, it is relatively easy and straightforward to implement the existing functions in hepfile to perform this conversion!