-
-
Notifications
You must be signed in to change notification settings - Fork 4.9k
mongodb native serialization and group results meta collection #1990
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…t collection than results and add a 'bson' fake kombu encoder to allow pymongo to serialize natively data in mongodb
|
Yes please. It would be nice for celery to have the option to store the results as native json and not binary into mongodb. There was a similar issue raised in the past but had no pull request #1200. |
|
This address the first request of #1200, the second one is not really needed, since you would need the object id nevertheless (or task id for results), so the best option is to save to a collection all the data we need before the task is started and reference this in the results with its object id. |
|
Thanks! Merged into master. I don't think you need to register a kombu serializer for this, so I changed it to override the backend methods instead. I also don't think one should expect to get stores group results by doing AsyncResult(group_id).get(). Can you also add your name to https://github.com/celery/celery/edit/master/CONTRIBUTORS.txt? |
|
Is there any reason this hasn't been backported to the 3.1 release? The part of the code that does fixes this issue: #2081 |
|
actually, this still doesn't quite fix it
|
|
@ask what are your thoughts on adding a configuration option allowing to store to content as a raw dict instead of doing |
a little more flexible approach on storing data effectively on mongodb allowing to map-reduce/aggregate later: