MySQL Cluster: The table is full












0















I have a problem with MySQL Cluster.



I created a table in order to be stored in the disk but i think it's always stored in memory.



I have have created a table with the query below:



   CREATE TABLE IF NOT EXISTS user (
`FIELD_KEY` varchar(255) NOT NULL,
`FIELD0` text,
`FIELD1` text,
`FIELD2` text,
`FIELD3` text,
`FIELD4` text,
`FIELD5` text,
`FIELD6` text,
`FIELD7` text,
`FIELD8` text,
`FIELD9` text,
PRIMARY KEY (YCSB_KEY)
) ENGINE=ndbcluster DEFAULT CHARSET=utf8 max_rows=40000000
TABLESPACE ts_1 STORAGE DISK;


ALTER TABLE usertable4M PARTITION BY KEY() PARTITIONS 4


My Environment (3 nodes in the same VM):



MySQL Cluster 7.5




  • management nodes: 1

  • data node: 1

  • sql nodes: 1


VM Config : 16GB RAM, 100GB HDD



MySQL Cluster settings (config.ini) are set to:



[ndbd default]
noofreplicas=1
DataMemory=2048M
# Memory to allocate for index storage
IndexMemory=1024M
# To support multi-thread processors
MaxNoOfExecutionThreads=2
FragmentLogFileSize=256M
NoOfFragmentLogFiles=12
#-------
RedoBuffer=32M
SharedGlobalMemory=256M
DiskPageBufferMemory=4096M
NoOfFragmentLogParts = 4
#-------
[ndb_mgmd]
HostName=192.168.1.10
DataDir=/var/lib/mysql-cluster

[ndbd]
HostName=192.168.1.10
NodeId=2
DataDir=/usr/local/mysql/data

[mysqld]
HostName=192.168.1.10


In order to store data in Disk i created a LOGFILE and TABLESPACE with the queries below:



CREATE LOGFILE GROUP lg ADD UNDOFILE 'undo1.dat'
INITIAL_SIZE=12288M
UNDO_BUFFER_SIZE=32M
ENGINE=NDB;

CREATE TABLESPACE ts_1
ADD DATAFILE 'data1.dat'
USE LOGFILE GROUP lg
INITIAL_SIZE=256M
ENGINE=NDB;

ALTER TABLESPACE ts_1 ADD DATAFILE 'data2.dat' INITIAL_SIZE=256M ENGINE=NDB;
.
.
.
ALTER TABLESPACE ts_1 ADD DATAFILE 'data24.dat' INITIAL_SIZE=256M ENGINE=NDB;


MemoryUsage report



$ sudo ndb_mgm -e 'all report MemoryUsage'
Connected to Management Server at: 192.168.1.10:1186
Node 2: Data usage is 94%(62259 32K pages of total 65536)
Node 2: Index usage is 2%(2801 8K pages of total 131104)


When I ran a query to insert 4000000 records (4GB) to my table I get the error (The table 'user' is full) and can not insert data any more to the table and only 1611787 records has been inserted . I don't understand why it is so.



Could somebody explain me what is the situation. And how can I resolve the problem.










share|improve this question



























    0















    I have a problem with MySQL Cluster.



    I created a table in order to be stored in the disk but i think it's always stored in memory.



    I have have created a table with the query below:



       CREATE TABLE IF NOT EXISTS user (
    `FIELD_KEY` varchar(255) NOT NULL,
    `FIELD0` text,
    `FIELD1` text,
    `FIELD2` text,
    `FIELD3` text,
    `FIELD4` text,
    `FIELD5` text,
    `FIELD6` text,
    `FIELD7` text,
    `FIELD8` text,
    `FIELD9` text,
    PRIMARY KEY (YCSB_KEY)
    ) ENGINE=ndbcluster DEFAULT CHARSET=utf8 max_rows=40000000
    TABLESPACE ts_1 STORAGE DISK;


    ALTER TABLE usertable4M PARTITION BY KEY() PARTITIONS 4


    My Environment (3 nodes in the same VM):



    MySQL Cluster 7.5




    • management nodes: 1

    • data node: 1

    • sql nodes: 1


    VM Config : 16GB RAM, 100GB HDD



    MySQL Cluster settings (config.ini) are set to:



    [ndbd default]
    noofreplicas=1
    DataMemory=2048M
    # Memory to allocate for index storage
    IndexMemory=1024M
    # To support multi-thread processors
    MaxNoOfExecutionThreads=2
    FragmentLogFileSize=256M
    NoOfFragmentLogFiles=12
    #-------
    RedoBuffer=32M
    SharedGlobalMemory=256M
    DiskPageBufferMemory=4096M
    NoOfFragmentLogParts = 4
    #-------
    [ndb_mgmd]
    HostName=192.168.1.10
    DataDir=/var/lib/mysql-cluster

    [ndbd]
    HostName=192.168.1.10
    NodeId=2
    DataDir=/usr/local/mysql/data

    [mysqld]
    HostName=192.168.1.10


    In order to store data in Disk i created a LOGFILE and TABLESPACE with the queries below:



    CREATE LOGFILE GROUP lg ADD UNDOFILE 'undo1.dat'
    INITIAL_SIZE=12288M
    UNDO_BUFFER_SIZE=32M
    ENGINE=NDB;

    CREATE TABLESPACE ts_1
    ADD DATAFILE 'data1.dat'
    USE LOGFILE GROUP lg
    INITIAL_SIZE=256M
    ENGINE=NDB;

    ALTER TABLESPACE ts_1 ADD DATAFILE 'data2.dat' INITIAL_SIZE=256M ENGINE=NDB;
    .
    .
    .
    ALTER TABLESPACE ts_1 ADD DATAFILE 'data24.dat' INITIAL_SIZE=256M ENGINE=NDB;


    MemoryUsage report



    $ sudo ndb_mgm -e 'all report MemoryUsage'
    Connected to Management Server at: 192.168.1.10:1186
    Node 2: Data usage is 94%(62259 32K pages of total 65536)
    Node 2: Index usage is 2%(2801 8K pages of total 131104)


    When I ran a query to insert 4000000 records (4GB) to my table I get the error (The table 'user' is full) and can not insert data any more to the table and only 1611787 records has been inserted . I don't understand why it is so.



    Could somebody explain me what is the situation. And how can I resolve the problem.










    share|improve this question

























      0












      0








      0








      I have a problem with MySQL Cluster.



      I created a table in order to be stored in the disk but i think it's always stored in memory.



      I have have created a table with the query below:



         CREATE TABLE IF NOT EXISTS user (
      `FIELD_KEY` varchar(255) NOT NULL,
      `FIELD0` text,
      `FIELD1` text,
      `FIELD2` text,
      `FIELD3` text,
      `FIELD4` text,
      `FIELD5` text,
      `FIELD6` text,
      `FIELD7` text,
      `FIELD8` text,
      `FIELD9` text,
      PRIMARY KEY (YCSB_KEY)
      ) ENGINE=ndbcluster DEFAULT CHARSET=utf8 max_rows=40000000
      TABLESPACE ts_1 STORAGE DISK;


      ALTER TABLE usertable4M PARTITION BY KEY() PARTITIONS 4


      My Environment (3 nodes in the same VM):



      MySQL Cluster 7.5




      • management nodes: 1

      • data node: 1

      • sql nodes: 1


      VM Config : 16GB RAM, 100GB HDD



      MySQL Cluster settings (config.ini) are set to:



      [ndbd default]
      noofreplicas=1
      DataMemory=2048M
      # Memory to allocate for index storage
      IndexMemory=1024M
      # To support multi-thread processors
      MaxNoOfExecutionThreads=2
      FragmentLogFileSize=256M
      NoOfFragmentLogFiles=12
      #-------
      RedoBuffer=32M
      SharedGlobalMemory=256M
      DiskPageBufferMemory=4096M
      NoOfFragmentLogParts = 4
      #-------
      [ndb_mgmd]
      HostName=192.168.1.10
      DataDir=/var/lib/mysql-cluster

      [ndbd]
      HostName=192.168.1.10
      NodeId=2
      DataDir=/usr/local/mysql/data

      [mysqld]
      HostName=192.168.1.10


      In order to store data in Disk i created a LOGFILE and TABLESPACE with the queries below:



      CREATE LOGFILE GROUP lg ADD UNDOFILE 'undo1.dat'
      INITIAL_SIZE=12288M
      UNDO_BUFFER_SIZE=32M
      ENGINE=NDB;

      CREATE TABLESPACE ts_1
      ADD DATAFILE 'data1.dat'
      USE LOGFILE GROUP lg
      INITIAL_SIZE=256M
      ENGINE=NDB;

      ALTER TABLESPACE ts_1 ADD DATAFILE 'data2.dat' INITIAL_SIZE=256M ENGINE=NDB;
      .
      .
      .
      ALTER TABLESPACE ts_1 ADD DATAFILE 'data24.dat' INITIAL_SIZE=256M ENGINE=NDB;


      MemoryUsage report



      $ sudo ndb_mgm -e 'all report MemoryUsage'
      Connected to Management Server at: 192.168.1.10:1186
      Node 2: Data usage is 94%(62259 32K pages of total 65536)
      Node 2: Index usage is 2%(2801 8K pages of total 131104)


      When I ran a query to insert 4000000 records (4GB) to my table I get the error (The table 'user' is full) and can not insert data any more to the table and only 1611787 records has been inserted . I don't understand why it is so.



      Could somebody explain me what is the situation. And how can I resolve the problem.










      share|improve this question














      I have a problem with MySQL Cluster.



      I created a table in order to be stored in the disk but i think it's always stored in memory.



      I have have created a table with the query below:



         CREATE TABLE IF NOT EXISTS user (
      `FIELD_KEY` varchar(255) NOT NULL,
      `FIELD0` text,
      `FIELD1` text,
      `FIELD2` text,
      `FIELD3` text,
      `FIELD4` text,
      `FIELD5` text,
      `FIELD6` text,
      `FIELD7` text,
      `FIELD8` text,
      `FIELD9` text,
      PRIMARY KEY (YCSB_KEY)
      ) ENGINE=ndbcluster DEFAULT CHARSET=utf8 max_rows=40000000
      TABLESPACE ts_1 STORAGE DISK;


      ALTER TABLE usertable4M PARTITION BY KEY() PARTITIONS 4


      My Environment (3 nodes in the same VM):



      MySQL Cluster 7.5




      • management nodes: 1

      • data node: 1

      • sql nodes: 1


      VM Config : 16GB RAM, 100GB HDD



      MySQL Cluster settings (config.ini) are set to:



      [ndbd default]
      noofreplicas=1
      DataMemory=2048M
      # Memory to allocate for index storage
      IndexMemory=1024M
      # To support multi-thread processors
      MaxNoOfExecutionThreads=2
      FragmentLogFileSize=256M
      NoOfFragmentLogFiles=12
      #-------
      RedoBuffer=32M
      SharedGlobalMemory=256M
      DiskPageBufferMemory=4096M
      NoOfFragmentLogParts = 4
      #-------
      [ndb_mgmd]
      HostName=192.168.1.10
      DataDir=/var/lib/mysql-cluster

      [ndbd]
      HostName=192.168.1.10
      NodeId=2
      DataDir=/usr/local/mysql/data

      [mysqld]
      HostName=192.168.1.10


      In order to store data in Disk i created a LOGFILE and TABLESPACE with the queries below:



      CREATE LOGFILE GROUP lg ADD UNDOFILE 'undo1.dat'
      INITIAL_SIZE=12288M
      UNDO_BUFFER_SIZE=32M
      ENGINE=NDB;

      CREATE TABLESPACE ts_1
      ADD DATAFILE 'data1.dat'
      USE LOGFILE GROUP lg
      INITIAL_SIZE=256M
      ENGINE=NDB;

      ALTER TABLESPACE ts_1 ADD DATAFILE 'data2.dat' INITIAL_SIZE=256M ENGINE=NDB;
      .
      .
      .
      ALTER TABLESPACE ts_1 ADD DATAFILE 'data24.dat' INITIAL_SIZE=256M ENGINE=NDB;


      MemoryUsage report



      $ sudo ndb_mgm -e 'all report MemoryUsage'
      Connected to Management Server at: 192.168.1.10:1186
      Node 2: Data usage is 94%(62259 32K pages of total 65536)
      Node 2: Index usage is 2%(2801 8K pages of total 131104)


      When I ran a query to insert 4000000 records (4GB) to my table I get the error (The table 'user' is full) and can not insert data any more to the table and only 1611787 records has been inserted . I don't understand why it is so.



      Could somebody explain me what is the situation. And how can I resolve the problem.







      mysql-cluster






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked 13 mins ago









      abouyahya85abouyahya85

      32




      32






















          0






          active

          oldest

          votes












          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "182"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f233457%2fmysql-cluster-the-table-is-full%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Database Administrators Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f233457%2fmysql-cluster-the-table-is-full%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Liste der Baudenkmale in Friedland (Mecklenburg)

          Single-Malt-Whisky

          Czorneboh