Loomio

mysql replication is not working (backup not uptodate)

Pirate Praveen
Pirate Praveen Public Seen by 363

We need to reinitialize the replication I think. The backup server was down, I restarted it, but replication is showing an error.

show slave status gave this error.

Last_IO_Errno: 1236
                Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'
               Last_SQL_Errno: 1594
               Last_SQL_Error: Relay log read failure: Could not parse relay log event entry. The possible reasons are: the master's binary log is corrupted (you can check this by running 'mysqlbinlog' on the binary log), the slave's relay log is corrupted (you can check this by running 'mysqlbinlog' on the relay log), a network problem, or a bug in the master's or slave's MySQL code. If you want to check the master's binary log or slave's relay log, you will be able to know their names by issuing 'SHOW SLAVE STATUS' on this slave.
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 3

Any volunteers to reset the replication? We'll have to copy the master db using rsync and restart the replication from new copy.

Pirate Praveen

Pirate Praveen January 17th, 2017 17:32

Some more info about the error https://www.percona.com/blog/2014/10/08/mysql-replication-got-fatal-error-1236-causes-and-cures/ basically we need to recreate replication as required log files got removed from master (older logs get cleared automatically).

DU

[deactivated account] January 20th, 2017 17:10

I will help in setting up the backup.

Pirate Praveen

Pirate Praveen January 21st, 2017 13:40

I have shared access with you. See http://fosscommunity.in/index.php?title=Poddery#Setting_up_Backup for steps to setup backup and also getting encrypted access file.

DU

[deactivated account] January 22nd, 2017 14:44

Completed backup. Slave is now up and running.

Pirate Praveen

Pirate Praveen January 22nd, 2017 14:53

Thanks for the help! Happy to see more hands helping. I have also added a ping test to uptime robot account so we will get notified if backup server goes down again. We will have to 10 days to fix things before the master server will clear the binary logs.

If you get a notification that Poddery Backup is down, just login to scaleway.com and see if the server is okay and "cold boot" if the server is hanged. We need to also find the root cause, @mintojoseph can you help?

Minto Joseph

Minto Joseph January 22nd, 2017 17:17

Will check this tomorrow.