Django: Save Unknown Object in Database

Implementing the procedures mentioned here in a production system may cause security issues in the future. I wrote this article as a hack.

Suppose you are dealing with a poorly written API, which returns different type of objects at different scenarios. For example, sometimes it returns a string(ie. abcd), sometimes it returns a list(ie. [1,2,3,4]), or maybe None. Now, you are suppose to save that data, which you can’t predict. As you can’t predict the data type of object, you can’t design your Django Model accordingly. How are you suppose to take care of that?

Convert to byte data

As you don’t know which data type you will be dealing with, so its safer to convert it to a common data type, which can be Byte Data. For that, you can use python pickle. For example:

import pickle

byte_data = pickle.dumps(obj)
obj = pickle.loads(byte_data)

Using ‘BinaryField’ to store in database

You can use Django BinaryField to store the byte data created using pickle. For example:

import pickle

class Parameter(models.Model):
    _value = models.BinaryField()

    def set_data(self, data):
        self._value = pickle.dumps(data)

    def get_data(self):
        return pickle.loads(self._value)

    value = property(get_data, set_data)

Usage examples

You can use the value from the example Model given in last step like this:

In: b = Foo.objects.create(value=1)

In: b.value
Out: 1

In: b = Foo.objects.create(value={1:1,2:2})

In: b.value
Out: {1: 1, 2: 2}

In: b = Foo.objects.create(value=[1,1])

In: b.value
Out: [1, 1]

Known issues

There are some known issues you need to know about. For example:

1. You can’t make db queries in this approach. Means you can’t do Foo.objects.get(value=1) or Foo.objects.filter(value=1) or any other DB query.

2. Also, you need to be careful about not abusing BinaryField. From Django Docs:

Although you might think about storing files in the database, consider that it is bad design in 99% of the cases. This field is not a replacement for proper static files handling.

Other information

You can look into my Stack Overflow Answer regarding this problem.

Last updated: Apr 05, 2024