Skip to content

Conversation

csirmazbendeguz
Copy link
Contributor

@csirmazbendeguz csirmazbendeguz commented Jul 23, 2024

Trac ticket number

ticket-373

Branch description

This is a prerequisite for composite primary keys (#18056).

  • related_lookups.MultiColSource -> expressions.Cols. Cols will also be used by composite primary keys.

  • Fixed esoteric ForeignObject "related" lookups.

    At the moment, the expression (a, b, c) < (x, y, z) compiles to:
    WHERE a < x AND b < y AND c < z.

    This is incorrect, e.g. (1, 9) < (2, 1) would evaluate as false.

    This PR addresses this bug by compiling the expression to:
    WHERE a < x OR (a = x AND (b < y OR (b = y AND c < z))) instead.

    If the backend supports tuple comparisons, it's compiled to:
    WHERE (a, b, c) < (x, y, z).

  • This fix is applied to <=, >, >= as well.

  • = will use tuple comparison if the backend supports it (WHERE (a, b, c) = (x, y, z)).

  • IN will use tuple comparison if the backend supports it ((a, b, c) IN ((x1, y1, z1), (x2, y2, z2))

  • IS NULL has no significant changes, only refactoring.

Checklist

  • This PR targets the main branch.
  • The commit message is written in past tense, mentions the ticket number, and ends with a period.
  • I have checked the "Has patch" ticket flag in the Trac system.
  • I have added or updated relevant tests.
  • I have added or updated relevant docs, including release notes if applicable.
  • I have attached screenshots in both light and dark modes for any UI changes.

@csirmazbendeguz csirmazbendeguz force-pushed the ticket_373_tuple_lookups branch from 36bd654 to 9b0fa13 Compare July 23, 2024 13:33
@csirmazbendeguz csirmazbendeguz force-pushed the ticket_373_tuple_lookups branch 2 times, most recently from 50d5d15 to 1a7d522 Compare July 23, 2024 17:16
@csirmazbendeguz csirmazbendeguz force-pushed the ticket_373_tuple_lookups branch from 3385c0b to 46bbb8e Compare July 24, 2024 16:28
@sarahboyce sarahboyce changed the title Fixed #373 - Added tuple lookups. Refs #373 -- Added tuple lookups. Jul 25, 2024
@csirmazbendeguz csirmazbendeguz force-pushed the ticket_373_tuple_lookups branch from 2f41437 to 0f23f0f Compare July 29, 2024 07:12
@csirmazbendeguz
Copy link
Contributor Author

csirmazbendeguz commented Aug 1, 2024

I changed as_sql based on @charettes 's and @felixxm 's feedback to use the tuple comparison syntax by default.

#18056 (comment)

@sarahboyce
Copy link
Contributor

buildbot, test on oracle.

@sarahboyce sarahboyce force-pushed the ticket_373_tuple_lookups branch from a3d0596 to 8fc4eb2 Compare August 1, 2024 13:30
@sarahboyce
Copy link
Contributor

buildbot, test on oracle.

Copy link
Contributor

@sarahboyce sarahboyce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this @csirmazbendeguz

@sarahboyce sarahboyce merged commit 1eac690 into django:main Aug 1, 2024
Comment on lines +26 to +57
def check_tuple_lookup(self):
assert isinstance(self.lhs, ColPairs)
self.check_rhs_is_tuple_or_list()
self.check_rhs_length_equals_lhs_length()

def check_rhs_is_tuple_or_list(self):
if not isinstance(self.rhs, (tuple, list)):
raise ValueError(
f"'{self.lookup_name}' lookup of '{self.lhs.field.name}' field "
"must be a tuple or a list"
)

def check_rhs_length_equals_lhs_length(self):
if len(self.lhs) != len(self.rhs):
raise ValueError(
f"'{self.lookup_name}' lookup of '{self.lhs.field.name}' field "
f"must have {len(self.lhs)} elements"
)

def check_rhs_is_collection_of_tuples_or_lists(self):
if not all(isinstance(vals, (tuple, list)) for vals in self.rhs):
raise ValueError(
f"'{self.lookup_name}' lookup of '{self.lhs.field.name}' field "
f"must be a collection of tuples or lists"
)

def check_rhs_elements_length_equals_lhs_length(self):
if not all(len(self.lhs) == len(vals) for vals in self.rhs):
raise ValueError(
f"'{self.lookup_name}' lookup of '{self.lhs.field.name}' field "
f"must have {len(self.lhs)} elements each"
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we don't have any tests for these checks? Secondly, some of them are used only in a single lookup so I'm not sure why there are in the mixin 🤔, and, as far as I'm aware, some of them are unnecessary, e.g. check_rhs_is_tuple_or_list().

We also should not call functions (e.g. len()) in f-strings as described in the contributing guide.

class ColPairs(Expression):
def __init__(self, alias, targets, sources, output_field):
super().__init__(output_field=output_field)
self.alias, self.targets, self.sources = alias, targets, sources
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't use this pattern, we removed it from some places in the past (e.g. 4e73d8c).

Check out #16805 (comment).

@csirmazbendeguz
Copy link
Contributor Author

csirmazbendeguz commented Aug 15, 2024

Thanks for the feedback @felixxm .

Why we don't have any tests for these checks?

As you noticed, these errors are mostly unreachable during normal code execution, they are more like sanity checks. Based on your feedback, I added tests for what I could Refs #373.

I also test them in the follow-up composite primary key PR more thoroughly Fixed #373.

so I'm not sure why there are in the mixin

Because I liked having all checks in one place... we can move it if you think this is a problem.

some of them are unnecessary

I don't think sanity checks are unnecessary.

should not call functions (e.g. len()) in f-strings

Ok, adjusted it in #18485.

@felixxm
Copy link
Member

felixxm commented Aug 15, 2024

As you noticed, these errors are mostly unreachable during normal code execution, they are more like sanity checks.

We normally don't add unreachable code to Django 😶

@csirmazbendeguz
Copy link
Contributor Author

@felixxm , let's discuss this in the new PR (#18485) please.

Comment on lines +81 to +84
@unittest.skipIf(
connection.vendor == "mysql",
"MySQL doesn't support LIMIT & IN/ALL/ANY/SOME subquery",
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of connection.vendor checks, please use DatabaseFeatures.django_test_skips instead.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We actually have a feature switch for that folks

# MySQL doesn't support sliced subqueries with IN/ALL/ANY/SOME.
allow_sliced_subqueries_with_in = False

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be handled by #18745

snorkelopstesting1-a11y pushed a commit to snorkel-marlin-repos/django_django_pr_18404_799b73ca-706a-4555-abfb-30103aef2598 that referenced this pull request Oct 11, 2025
Original PR #18404 by csirmazbendeguz
Original: django/django#18404
snorkelopstesting1-a11y added a commit to snorkel-marlin-repos/django_django_pr_18404_799b73ca-706a-4555-abfb-30103aef2598 that referenced this pull request Oct 11, 2025
snorkelopstesting1-a11y pushed a commit to snorkel-marlin-repos/django_django_pr_18404_bcf06d08-e4df-49aa-8e57-a279a0097732 that referenced this pull request Oct 11, 2025
Original PR #18404 by csirmazbendeguz
Original: django/django#18404
snorkelopstesting1-a11y added a commit to snorkel-marlin-repos/django_django_pr_18404_bcf06d08-e4df-49aa-8e57-a279a0097732 that referenced this pull request Oct 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants