datetime.date(2008, 5, 27) is not JSON serializable

[n3storm]

I get this error while requesting a list of resources:

´´´
127.0.0.1 - - [23/Dec/2013 14:11:58] "GET /entities HTTP/1.1" 500 -
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1836, in call
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functionsrule.endpoint
File "/usr/local/lib/python2.7/dist-packages/sandman/sandman.py", line 397, in get_collection
return collection_response(resources)
File "/usr/local/lib/python2.7/dist-packages/sandman/sandman.py", line 228, in collection_response
return _collection_json_response(resources)
File "/usr/local/lib/python2.7/dist-packages/sandman/sandman.py", line 101, in _collection_json_response
return jsonify(resources=result_list)
File "/usr/local/lib/python2.7/dist-packages/flask/json.py", line 238, in jsonify
indent=indent),
File "/usr/local/lib/python2.7/dist-packages/flask/json.py", line 126, in dumps
rv = _json.dumps(obj, **kwargs)
File "/usr/share/pyshared/simplejson/init.py", line 334, in dumps
**kw).encode(obj)
File "/usr/share/pyshared/simplejson/encoder.py", line 239, in encode
chunks = list(chunks)
File "/usr/share/pyshared/simplejson/encoder.py", line 551, in _iterencode
for chunk in _iterencode_dict(o, _current_indent_level):
File "/usr/share/pyshared/simplejson/encoder.py", line 514, in _iterencode_dict
for chunk in chunks:
File "/usr/share/pyshared/simplejson/encoder.py", line 422, in _iterencode_list
for chunk in chunks:
File "/usr/share/pyshared/simplejson/encoder.py", line 514, in _iterencode_dict
for chunk in chunks:
File "/usr/share/pyshared/simplejson/encoder.py", line 561, in _iterencode
o = _default(o)
File "/usr/local/lib/python2.7/dist-packages/flask/json.py", line 83, in default
return _json.JSONEncoder.default(self, o)
File "/usr/share/pyshared/simplejson/encoder.py", line 213, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: datetime.date(2008, 5, 27) is not JSON serializable
´´´

Database field is of datetime type.
Listing an editing through admin interface works ok.

Cheers.

[jeffknupp]

Let me quickly create a test database to recreate the issue. You wouldn't happen to have your database creation SQL lying around anywhere, would you?

[n3storm]

Its a 356 fields table (glups) with real customer data 13 954 rows (not that much).

If you I agree I will try to replicate with a smaller version of the table and see what happens.

[jeffknupp]

No, that's fine. I'll create my own on a local mysql instance. Out of curiosity, what database engine were you using?

[jeffknupp]

I've found that this usually happens when a "DateTime" column is actually only storing a "Date" (i.e. MM-DD-YYYY instead of MM-DD-YYYY HH:mm:SS). If that's the case in your scenario, it looks like a deficiency of the SQLAlchemy introspection portion. Could you check if that's the case.

Regardless, I'll open a ticket with them, as this should be supported (if the databases support it, which they do)

[n3storm]

That's right, all rows in that column are storing MM-DD-YYYY and it fails since first row!
If I add HH:mm:SS to the cell, it fails on next row so it's almost confirmed you are right.

[dolinsky]

Not sure if this has been figured out yet or not, but I encountered the same issue (we have both date and datetime columns in our PG dbs). The answer was in the flask.json_encoder class, which extends the simplejson encoder and adds support for datetime, but not date. I overrode the default method with the following:

#!/usr/bin/env python

from sandman import app
from flask.json import JSONEncoder

class CustomJSONEncoder(JSONEncoder):
    def default(self, obj):
        try:
            if isinstance(obj, date):
                return obj.isoformat()
            iterable = iter(obj)
        except TypeError:
            pass
        else:
            return list(iterable)
        return JSONEncoder.default(self, obj)

app.json_encoder = CustomJSONEncoder

https://gist.github.com/dolinsky/9124542

[jeffknupp]

Thanks for the info. That may mean the solution is as simple as not using
Flask's jsonify and just using the builtin json.dumps instead.

[mritunjay10]

Not sure if this has been figured out yet or not, but I encountered the same issue (we have both date and datetime columns in our PG dbs). The answer was in the flask.json_encoder class, which extends the simplejson encoder and adds support for datetime, but not date. I overrode the default method with the following:

#!/usr/bin/env python

from sandman import app
from flask.json import JSONEncoder

class CustomJSONEncoder(JSONEncoder):
    def default(self, obj):
        try:
            if isinstance(obj, date):
                return obj.isoformat()
            iterable = iter(obj)
        except TypeError:
            pass
        else:
            return list(iterable)
        return JSONEncoder.default(self, obj)

app.json_encoder = CustomJSONEncoder

https://gist.github.com/dolinsky/9124542

Thanks, this worked for me!