Backing up mongo database of about 120 GB size
We have a mongo database of about 120 GB size. I have run mongodump using nohup and redirecting the logs to /dev/null about 3 days back, but the dump file is ~40GB in size now, and the dump is still running. Is this expected?
If yes, what is the approximate compression ratio for a mongo database? i.e. for a 120 GB database, how much is the backup file size going to be?
This would help me in estimating the time remaining for the dump to finish. I have no clue why it is taking up so much time, also, wanted to know if there is a faster/better way of backing up the mongo database (remote copy is not something i'm considering)?
We are running this on the live system, but are currently not using this database. So, effectively, mongodump is the only client that mongod is serving.
mongodb
edited 9 mins ago
Paul White♦
51k14278450
asked Dec 21 '15 at 18:33
sreeraagsreeraag
115
add a comment |
We have a mongo database of about 120 GB size. I have run mongodump using nohup and redirecting the logs to /dev/null about 3 days back, but the dump file is ~40GB in size now, and the dump is still running. Is this expected?
If yes, what is the approximate compression ratio for a mongo database? i.e. for a 120 GB database, how much is the backup file size going to be?
This would help me in estimating the time remaining for the dump to finish. I have no clue why it is taking up so much time, also, wanted to know if there is a faster/better way of backing up the mongo database (remote copy is not something i'm considering)?
We are running this on the live system, but are currently not using this database. So, effectively, mongodump is the only client that mongod is serving.
mongodb
edited 9 mins ago
Paul White♦
51k14278450
asked Dec 21 '15 at 18:33
sreeraagsreeraag
115
We need way more information. OS? Filesystem? Piping to compressor? Inserts,updates, deletes/second? Sharded or not? HDD or SDD? Without log files, we can of course not see wether there are problems. Are you backing up to the same disk where your dbpath points to? Give us more hard facts, please.
– Markus W Mahlberg
Dec 27 '15 at 0:42
add a comment |
We have a mongo database of about 120 GB size. I have run mongodump using nohup and redirecting the logs to /dev/null about 3 days back, but the dump file is ~40GB in size now, and the dump is still running. Is this expected?
If yes, what is the approximate compression ratio for a mongo database? i.e. for a 120 GB database, how much is the backup file size going to be?
This would help me in estimating the time remaining for the dump to finish. I have no clue why it is taking up so much time, also, wanted to know if there is a faster/better way of backing up the mongo database (remote copy is not something i'm considering)?
We are running this on the live system, but are currently not using this database. So, effectively, mongodump is the only client that mongod is serving.
mongodb
edited 9 mins ago
Paul White♦
51k14278450
asked Dec 21 '15 at 18:33
sreeraagsreeraag
115
We have a mongo database of about 120 GB size. I have run mongodump using nohup and redirecting the logs to /dev/null about 3 days back, but the dump file is ~40GB in size now, and the dump is still running. Is this expected?
If yes, what is the approximate compression ratio for a mongo database? i.e. for a 120 GB database, how much is the backup file size going to be?
This would help me in estimating the time remaining for the dump to finish. I have no clue why it is taking up so much time, also, wanted to know if there is a faster/better way of backing up the mongo database (remote copy is not something i'm considering)?
We are running this on the live system, but are currently not using this database. So, effectively, mongodump is the only client that mongod is serving.
mongodb
mongodb
edited 9 mins ago
Paul White♦
51k14278450
asked Dec 21 '15 at 18:33
sreeraagsreeraag
115
edited 9 mins ago
Paul White♦
51k14278450
asked Dec 21 '15 at 18:33
sreeraagsreeraag
115
edited 9 mins ago
Paul White♦
51k14278450
edited 9 mins ago
Paul White♦
51k14278450
edited 9 mins ago
Paul White♦
51k14278450
51k14278450
asked Dec 21 '15 at 18:33
sreeraagsreeraag
115
asked Dec 21 '15 at 18:33
sreeraagsreeraag
115
asked Dec 21 '15 at 18:33
sreeraagsreeraag
115
115
We need way more information. OS? Filesystem? Piping to compressor? Inserts,updates, deletes/second? Sharded or not? HDD or SDD? Without log files, we can of course not see wether there are problems. Are you backing up to the same disk where your dbpath points to? Give us more hard facts, please.
– Markus W Mahlberg
Dec 27 '15 at 0:42
add a comment |
We need way more information. OS? Filesystem? Piping to compressor? Inserts,updates, deletes/second? Sharded or not? HDD or SDD? Without log files, we can of course not see wether there are problems. Are you backing up to the same disk where your dbpath points to? Give us more hard facts, please.
– Markus W Mahlberg
Dec 27 '15 at 0:42
We need way more information. OS? Filesystem? Piping to compressor? Inserts,updates, deletes/second? Sharded or not? HDD or SDD? Without log files, we can of course not see wether there are problems. Are you backing up to the same disk where your dbpath points to? Give us more hard facts, please.
– Markus W Mahlberg
Dec 27 '15 at 0:42
We need way more information. OS? Filesystem? Piping to compressor? Inserts,updates, deletes/second? Sharded or not? HDD or SDD? Without log files, we can of course not see wether there are problems. Are you backing up to the same disk where your dbpath points to? Give us more hard facts, please.
– Markus W Mahlberg
Dec 27 '15 at 0:42
add a comment |
1 Answer
1
active
oldest
votes
Answers originally left in comments:
Compressed size will depend on how compressible your data patterns inside your documents and collections are. Mongo isn't going to backup the indexes so subtract whatever indexes you have from that 120GB and that's what it will backup, along with any profiling data you have saved in the DB itself. There's a lot to consider when backing up mongo. V3 and 2.8 backup methods are similar. – Ali Razeghi
If you're not using the database, you could just stop the mongo service, and make a copy of the data directory - if you update on the same server. You stop mongo service, read upgrade info on mongo website. And start your new mongo, it will use your existing data directory.
If you do the same on another server, it's like an upgrade. In your script, save the mongo version, and install the same mongo version:
sudo apt-get install -y
mongodb-org=3.0.8 mongodb-org-server=3.0.8 mongodb-org-shell=3.0.8
mongodb-org-mongos=3.0.8 mongodb-org-tools=3.0.8
...and run an upgrade afterwards. Always check upgrade info on the mongo website! – aldwinaldwin
answered 9 mins ago
Comment ConverterComment Converter
1,1251323
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "182"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f124271%2fbacking-up-mongo-database-of-about-120-gb-size%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Answers originally left in comments:
Compressed size will depend on how compressible your data patterns inside your documents and collections are. Mongo isn't going to backup the indexes so subtract whatever indexes you have from that 120GB and that's what it will backup, along with any profiling data you have saved in the DB itself. There's a lot to consider when backing up mongo. V3 and 2.8 backup methods are similar. – Ali Razeghi
If you're not using the database, you could just stop the mongo service, and make a copy of the data directory - if you update on the same server. You stop mongo service, read upgrade info on mongo website. And start your new mongo, it will use your existing data directory.
If you do the same on another server, it's like an upgrade. In your script, save the mongo version, and install the same mongo version:
sudo apt-get install -y
mongodb-org=3.0.8 mongodb-org-server=3.0.8 mongodb-org-shell=3.0.8
mongodb-org-mongos=3.0.8 mongodb-org-tools=3.0.8
...and run an upgrade afterwards. Always check upgrade info on the mongo website! – aldwinaldwin
answered 9 mins ago
Comment ConverterComment Converter
1,1251323
add a comment |
Answers originally left in comments:
Compressed size will depend on how compressible your data patterns inside your documents and collections are. Mongo isn't going to backup the indexes so subtract whatever indexes you have from that 120GB and that's what it will backup, along with any profiling data you have saved in the DB itself. There's a lot to consider when backing up mongo. V3 and 2.8 backup methods are similar. – Ali Razeghi
If you're not using the database, you could just stop the mongo service, and make a copy of the data directory - if you update on the same server. You stop mongo service, read upgrade info on mongo website. And start your new mongo, it will use your existing data directory.
If you do the same on another server, it's like an upgrade. In your script, save the mongo version, and install the same mongo version:
sudo apt-get install -y
mongodb-org=3.0.8 mongodb-org-server=3.0.8 mongodb-org-shell=3.0.8
mongodb-org-mongos=3.0.8 mongodb-org-tools=3.0.8
...and run an upgrade afterwards. Always check upgrade info on the mongo website! – aldwinaldwin
answered 9 mins ago
Comment ConverterComment Converter
1,1251323
add a comment |
Answers originally left in comments:
Compressed size will depend on how compressible your data patterns inside your documents and collections are. Mongo isn't going to backup the indexes so subtract whatever indexes you have from that 120GB and that's what it will backup, along with any profiling data you have saved in the DB itself. There's a lot to consider when backing up mongo. V3 and 2.8 backup methods are similar. – Ali Razeghi
If you're not using the database, you could just stop the mongo service, and make a copy of the data directory - if you update on the same server. You stop mongo service, read upgrade info on mongo website. And start your new mongo, it will use your existing data directory.
If you do the same on another server, it's like an upgrade. In your script, save the mongo version, and install the same mongo version:
sudo apt-get install -y
mongodb-org=3.0.8 mongodb-org-server=3.0.8 mongodb-org-shell=3.0.8
mongodb-org-mongos=3.0.8 mongodb-org-tools=3.0.8
...and run an upgrade afterwards. Always check upgrade info on the mongo website! – aldwinaldwin
answered 9 mins ago
Comment ConverterComment Converter
1,1251323
Answers originally left in comments:
Compressed size will depend on how compressible your data patterns inside your documents and collections are. Mongo isn't going to backup the indexes so subtract whatever indexes you have from that 120GB and that's what it will backup, along with any profiling data you have saved in the DB itself. There's a lot to consider when backing up mongo. V3 and 2.8 backup methods are similar. – Ali Razeghi
If you're not using the database, you could just stop the mongo service, and make a copy of the data directory - if you update on the same server. You stop mongo service, read upgrade info on mongo website. And start your new mongo, it will use your existing data directory.
If you do the same on another server, it's like an upgrade. In your script, save the mongo version, and install the same mongo version:
sudo apt-get install -y
mongodb-org=3.0.8 mongodb-org-server=3.0.8 mongodb-org-shell=3.0.8
mongodb-org-mongos=3.0.8 mongodb-org-tools=3.0.8
...and run an upgrade afterwards. Always check upgrade info on the mongo website! – aldwinaldwin
answered 9 mins ago
Comment ConverterComment Converter
1,1251323
answered 9 mins ago
Comment ConverterComment Converter
1,1251323
answered 9 mins ago
Comment ConverterComment Converter
1,1251323
answered 9 mins ago
Comment ConverterComment Converter
1,1251323
1,1251323
add a comment |
add a comment |
Thanks for contributing an answer to Database Administrators Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f124271%2fbacking-up-mongo-database-of-about-120-gb-size%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
We need way more information. OS? Filesystem? Piping to compressor? Inserts,updates, deletes/second? Sharded or not? HDD or SDD? Without log files, we can of course not see wether there are problems. Are you backing up to the same disk where your dbpath points to? Give us more hard facts, please.
– Markus W Mahlberg
Dec 27 '15 at 0:42