Optimizing for Modulo Queries
I have a use case where I want to be able to query tens of thousands of records for the modulo of a certain field value with another number, i.e. queries like
SELECT * FROM EVENTS WHERE MOD(OFFSET)=5
What are some performance considerations, if any, I should have in place to ensure that my queries are more efficient? Are there any specific kinds of indexes that would speed up this kind of querying?
Has there been a published comparative analysis or benchmark or some kind that has been done for these kinds of queries between various databases? Alternatively, is there a naive way of determining what choice of database would be best suited for this kind of querying?
mysql postgresql
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
bumped to the homepage by Community♦ 42 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
I have a use case where I want to be able to query tens of thousands of records for the modulo of a certain field value with another number, i.e. queries like
SELECT * FROM EVENTS WHERE MOD(OFFSET)=5
What are some performance considerations, if any, I should have in place to ensure that my queries are more efficient? Are there any specific kinds of indexes that would speed up this kind of querying?
Has there been a published comparative analysis or benchmark or some kind that has been done for these kinds of queries between various databases? Alternatively, is there a naive way of determining what choice of database would be best suited for this kind of querying?
mysql postgresql
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
bumped to the homepage by Community♦ 42 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
I have a use case where I want to be able to query tens of thousands of records for the modulo of a certain field value with another number, i.e. queries like
SELECT * FROM EVENTS WHERE MOD(OFFSET)=5
What are some performance considerations, if any, I should have in place to ensure that my queries are more efficient? Are there any specific kinds of indexes that would speed up this kind of querying?
Has there been a published comparative analysis or benchmark or some kind that has been done for these kinds of queries between various databases? Alternatively, is there a naive way of determining what choice of database would be best suited for this kind of querying?
mysql postgresql
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
I have a use case where I want to be able to query tens of thousands of records for the modulo of a certain field value with another number, i.e. queries like
SELECT * FROM EVENTS WHERE MOD(OFFSET)=5
What are some performance considerations, if any, I should have in place to ensure that my queries are more efficient? Are there any specific kinds of indexes that would speed up this kind of querying?
Has there been a published comparative analysis or benchmark or some kind that has been done for these kinds of queries between various databases? Alternatively, is there a naive way of determining what choice of database would be best suited for this kind of querying?
mysql postgresql
mysql postgresql
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
1011
bumped to the homepage by Community♦ 42 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 42 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "182"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f84841%2foptimizing-for-modulo-queries%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
edited Dec 10 '14 at 20:40
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
93.6k9180294
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
Thanks for contributing an answer to Database Administrators Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f84841%2foptimizing-for-modulo-queries%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
