Skip to content

Conversation

@rtritto
Copy link
Member

@rtritto rtritto commented Dec 22, 2022

This change is Reviewable


Changes

  • As default, use find instead of aggregate (fix Cannot open large collections #1086).
  • Add countDocuments ( doc ) to count all documents.
  • Handle simple (Object query for $match) and complex pipeline (Array pipeline) when using advanced query in UI
  • $facet is moved only to complex pipeline (Array pipeline) and not to default query (that will fix Cannot open collections on MongoDB Version 4.2.22 #1082).
  • Add Promise.all to get stats and indexes (removed useless check for no collection found)

@rtritto
Copy link
Member Author

rtritto commented Dec 22, 2022

@armellarcier
Copy link
Contributor

I'm following your commits. Looks great, thanks!

@armellarcier
Copy link
Contributor

You removed a "collection not found" error handling, is it handled somewhere else?

@rtritto
Copy link
Member Author

rtritto commented Dec 22, 2022

The exsitence check of a collection is here:

/lib/router.js

    if (!mongo.collections[req.dbName].includes(id)) {
      req.session.error = 'Collection not found!';
      return res.redirect(res.locals.baseHref + 'db/' + req.dbName);
    }

I think that if a collection exists, it also have the stats and, for a collection not found, there's a redirect in router (/lib/routes/collection.js isn't called).

@shakaran shakaran added this to the 1.1 milestone Dec 22, 2022
Copy link
Collaborator

@shakaran shakaran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r2, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @rtritto)

@rtritto rtritto merged commit bd56ac5 into master Dec 23, 2022
@rtritto rtritto deleted the improve-find-aggregate branch December 23, 2022 00:01
@dariowrk
Copy link

Hi rtritto!

Thanks for all the fixes these last days. Amazing work, really appreciate it! Everything seems to work now. (Opening documents with Base64 BinaryIDs, Opening collections in Mongo 4.X and Opening rather large collections)

Regarding:
As default, use find instead of aggregate (fix #1086).

This works, i can open the large collections now. It's just slow as heck, i remember with a version of mongo-express a few months back (1.0.0-alpha4 IIRC) it was rather quick.

To open or change page in a collection of 5.666.031 documents it takes about 50 seconds in the most recent build.

@rtritto
Copy link
Member Author

rtritto commented Dec 23, 2022

@dariowrk created #1090

@rtritto
Copy link
Member Author

rtritto commented Dec 23, 2022

This works, i can open the large collections now. It's just slow as heck, i remember with a version of mongo-express a few months back (1.0.0-alpha4 IIRC) it was rather quick.

@dariowrk the 1.0.0-alpha4 use the count that is deprecated.
It's replaced with countDocuments and estimatedDocumentCount (source).

Under the hood, count uses find while countDocuments uses aggregate that is slower.

Mamoona-Ghania pushed a commit to FaizaAslam013/Devops-Mids-Exam that referenced this pull request Dec 13, 2025
* Improve find and aggregate

* Fix query of countDocuments and split pipelines of aggregates
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot open large collections Cannot open collections on MongoDB Version 4.2.22

5 participants