Read and Write JSON File Python 3

JSON data is a pretty common format, especially if you work with API. Many popular APIs will give or expect to get data in JSON format. Here is how to read and write JSON data files with Python 3.

What is JSON?

In the most simple form, JSON is a text file containing data. You can however, have very complicated data structures!

List of JSON Data Formats

JSON can be any data format that you choose (some examples):

  • Strings
  • Lists/Tuples
  • Dictionaries
  • List of Lists
  • Lists of Dictionaries

Whatever your data structure is without JSON, it will remain the same with JSON. JSON is just a way to represent the data, in a text file.

Using Python 3 to Write JSON

Here are some examples of using python JSON module to write sample JSON for you (and their corresponding JSON examples).

Sample Code to Create JSON Python3 String

Creating a string as a JSON file is hardly worth the effort, but in case you wanted to script it, here it is:

String Sample Output:

Sample Code to Create JSON Python3 List

Here is the Python 3 Script for making a list, and writing it out to a JSON data file:

List Sample Output:

Sample Code to Create JSON Python3 Tuple

Here is the Python 3 Script for making a tuple and writing it out to a JSON data file:

Sample output of a tuple:

If you are paying attention, you’ll see that both a list and a tuple look the same in JSON. JSON doesn’t make it a list or a tuple, how you load and use it does!

Sample Code to Create JSON Python3 Dictionary

Dictionaries are quite common when working with JSON, especially as building blocks for more complicated data structures. Here is a python script to create a dictionary file in JSON:

Output sample for a JSON dictionary:

Sample Code to Create JSON Python3 List of Lists

If you wanted to make a list of lists (a matrix) in python and create a json data file, you could do something like this:

Output example of JSON data for a list of lists in JSON:

Sample Code to Create JSON Python3 Nested Tuple

Nested tuples can be created in JSON using python just as simple as any other data format:

Output of a nested tuple in JSON:

Using JSON Data Like a Table (List of Dictionaries)

For the final example, I’ll share how to manipulate JSON data that mimics a table. For this example, I am writing a script to talk to the API on a popular firewall. My script needs to know my current IP (a dynamic DNS). Once I know my IP, then I can open up the firewall. As a “first step”, I needed to have a rule base so I could manage my firewall. This script example is that “first step”.

Common ports I will be working with are 22,80,443:

  • tcp/22 (ssh)
  • tcp/80 (http)
  • tcp/443 (https)

Here is the sample data I want to load as an initial rule set using those common ports and a structure similar to what I will work with in the final script:

ID Source IP Dest IP Service Description
100 1.2.3.4 2.3.4.5 tcp/22 Data Center A
101 1.2.3.4 2.3.4.5 tcp/80 Data Center B
102 1.2.3.4 2.3.4.5 tcp/443 Data Center B

My list of dictionaries uses the same keys as part of my logic, like a table header (see example above). I wanted to represent a much more complicated rule set in a spreadsheet for easy human readable audits but use scripting to talk to the API.

In this example we are using a data structure in Python that is a list of dictionaries and I wasn’t quite sure what it was supposed to look like.

The quick solution (instead of looking up data structures), was to just print it out and let Python tell me what it was supposed to look like, so that’s what I did.

Here is the script that printed out my data and json for me using sample data:

Here is how the JSON should look for a list of dictionaries:

Once I saw what it was supposed to look like I could then verify the rest of my code.

Reading JSON Data in Python

If we save this as a simple text file such as “rulesin.json” we can then read and update that file using Python. Reading is done withe ‘r’, just like writing is done with the ‘w’ flag. The following example loads my json file into ‘data’ variable. Because the JSON data is a list of dictionaries, ‘data’ will also be a list of dictionaries and can be used the same way.

Now all of the table is loaded into the “data” list of dictionaries. Here is how to iterate over each list member and access that data. Note, I am using rid, src, dst, svc, and des as my dictionary key names and accessing those named members using the “i” variable in a loop, of the enumerated “data” list of dictionaries in my example above. Your exact data structure doesn’t matter, as long as the JSON matches what you expect.

Looping Over a List of Dictionaries in Python 3

Now we can print out the data if we needed (or do other operations with it). In the following example code I am using the \t (tab) to display my output on a command line:

You could of course do anything you want to the data, munge it, consume it, etc. I wanted to update the source ip so I added another small function and some error checking:

Final Script: Get IP and Update JSON file

Combining all of it together is the script that will do what I wanted:

  • Read from rulesin.json file
  • Get my public IP
  • Update data with my Current IP
  • Write updated data to the rulesout.json file

The output of this program simply prints what it reads from the JSON file, after modifying the source IP:

And the updated json data:

Common Problems with JSON

JSON is finicky. If you don’t have your commas right, spacing, or other characters just right it could fail to read. You can Validate JSON using any of the many free tools online. Another option is to use python to write out your JSON file data. Python will write JSON just like it sees the data – and sometimes this will reveal your typos. Keep these tips in mind:

  • Properly Nest all JSON Data
  • Use commas properly, especially at the end of a list
  • Do not mix tabs and spaces
  • Do not comment your JSON using # or // type of comments
  • Remember JSON is a text file, not a database!

The Wrong Way to Use JSON and Python

JSON is an established file format but should not be used for extensive data entries where tens of thousands of records are required or where you expect heavy disk I/O (obviously). For that type of data, look towards proper database setups! However, for smaller records it works great.

Problem Solving with Python

In this article, you have learned quite a bit about reading, writing and even validating json files, but you also learned about various data structures, looping, and getting your public IP using a definition.

Now that you know how to do these things in Python, what will you do with your programming skill? Share your ideas below!

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.