Skip to main content

Load Data Infile - MySQL

Load data infile will read the content from the text file and insert into table. This process is very fast in MySQL.

Below is the Syntax suggested by MySQL in their documentation:
LOAD DATA [LOW_PRIORITY | CONCURRENT] [LOCAL] INFILE 'file_name'
    [REPLACE | IGNORE]
    INTO TABLE tbl_name
    [PARTITION (partition_name,...)]
    [CHARACTER SET charset_name]
    [{FIELDS | COLUMNS}
        [TERMINATED BY 'string']
        [[OPTIONALLY] ENCLOSED BY 'char']
        [ESCAPED BY 'char']
    ]
    [LINES
        [STARTING BY 'string']
        [TERMINATED BY 'string']
    ]
    [IGNORE number {LINES | ROWS}]
    [(col_name_or_user_var,...)]
    [SET col_name = expr,...]
Will look into some of the procedures to load data into table using an external text file:

Procedure 1:

Text file contains header:
LOAD DATA INFILE ‘path/to/example.csv’ 
INTO TABLE example FIELDS TERMINATED BY ‘,’ 
LINES TERMINATED BY ‘\n’ IGNORE 1 LINES;
Above, IGNORE 1 LINES considers that you have a header in your actual text file.

Similarly, if your text file doesn't contains any header then you can remove the  IGNORE 1 LINES part from the actual query and run.

Procedure 2:

Text file contains different sequence of columns when compared with table.

In this case compare the columns with the file and provide the field names in the query what will match to that in the text file respectively.
LOAD DATA INFILE ‘path/to/example.csv’ 
INTO TABLE example FIELDS TERMINATED BY ‘,’ 
LINES TERMINATED BY ‘\n’ IGNORE 1 LINES
(column_1,column_3,column_5,column_2);
What if the sequential order is different in table and text file, we don't need to about this case

Let's say in your text file column_1 is in fifth position, then provide the columns which actually are there in the text file. Here you need to follow the text file sequence instead of table.

Comments


  1. I have a doubt, I am trying to do everything as the tutorial tells me but the problem is that only mySQL assumes the first line of the file, I cannot insert the missing 8562 lines of the document.

    ReplyDelete

Post a Comment

Popular posts from this blog

Warning: Using a password on the command line interface can be insecure

We usually use MySQL login information on shell scripts to load some data into database or to fetch some monitoring information. To login to the database will write login information on Shell Scripts as below #!/bin/sh username = someusername password = somepassword $ mysql -u$username -p$password -A -v -S By doing that so, we will hit a warning in stdout saying: Warning: Using a password on the command line interface can be insecure To avoid the warning in shell script, we need to so some workaround like below: 1. Create a hidden file configuration file > touch /tmp/.mysqllogin.cnf 2. Enter your database credential information on the hidden file: > vi /tmp/.mysqllogin.cnf [client] user=someusername password=somepassword 3. Save the file and try to login to mysql using  --defaults-extra-file  keyword like $ mysql --defaults-extra-file=/tmp/.mysqllogin.cnf -A -v By using this workaround you can avoid the warning.

Unable to open the log file (mysqld)

While starting the server you may find the below errors sometimes, it's because of unintentionally deleting the MySQL Binary Log files. When starting mysql if the server unable to find the mysql-bin-log files then below errors will arise. 170418 12:15:19 InnoDB: 5.5.30 started; log sequence number 32347593422 /usr/sbin/mysqld: File './mysql-bin.000023' not found (Errcode: 2) 170418 12:15:19 [ERROR] Failed to open log (file './mysql-bin.000023', errno 2) 170418 12:15:19 [ERROR] Could not open log file 170418 12:15:19 [ERROR] Can't init tc log 170418 12:15:19 [ERROR] Aborting 170418 12:15:19 InnoDB: Starting shutdown... 170418 12:15:20 InnoDB: Shutdown completed; log sequence number 32347593422 170418 12:15:20 [Note] /usr/sbin/mysqld: Shutdown complete Do we have a solution to get rid of this and start MySQL again? Well, we have two types of workarounds to start MySQL again. 1. Deleting the missed files in binlog .index  file 2. Disabling the binlog log

Monitoring MySQL with Percona Monitoring and Management

In this blog post we will see how to monitor MySQL databases using the open source monitoring tool provided by Percona, referred as Percona Monitoring and Management (PMM). About PMM: Percona Monitoring and Management (PMM) is an open-source platform for managing and monitoring MySQL and MongoDB performance. It is developed by Percona in collaboration with experts in the field of managed database services, support and consulting. PMM is a free and open-source solution that you can run in your own environment for maximum security and reliability. It provides thorough time-based analysis for MySQL and MongoDB servers to ensure that your data works as efficiently as possible. Basic Requirements: 1. Need at least one Linux server to install PMM Client and PMM Server. 2. Need Operating System - root user credentials 3. Default port's(80) have to be open if using to or more servers. 4. A MySQL user is required to capture the Queries in the Database. 5. A Docker se