HBase Operation and Maintenance】Summary of non-stop service migration scenarios

Migration is a family affair for O&M, and non-stop service migration is an important part of theDBAof the ordeal, the following summarizes what is currently known about the migration program:

Utilizing the replication andsnapshot Complete data migration, automatic business switchover

Steps.

1. Create a replication of the two clusters, master and slave (the cluster to be migrated ----> the new cluster)
2. Create the same tables in the new cluster as in the old one, and start synchronizing them.
3. Pause the synchronization, use snapshot , make a snapshot of the old cluster table, for example table A

list_snapshots



     snapshot'A', 'A_snapshot'




     hbase -snapshot A_snapshot -copy-to hdfs://new cluster:8020/hbase -mappers

Once this is done, import snaphsot in the new cluster

disable ‘A'




      restore_snapshot ‘A_snapshot’



      enable ‘A'

4. Continue Synchronization, Complete Migration, Business Selection Time Migration

Utilizing bulkload

Create a new table in the old cluster, cut traffic, use bulkload to replenish data

The basic idea is to first copy and replicate the snapshot to the destination cluster, then turn on real-time synchronization of data between the two clusters, and finally, create the snapshot to turn on thedata synchronizationThe data between them is copied through the MR task.

Take the table test_table as an example:

1. Add the REPLICATION relationship from the original cluster to the target cluster and create the same tables in the target cluster as in the original cluster;

2. Import historical data into the slave cluster via snapshot: Execute the following command in the original cluster:

a、list_snapshots



b、snapshot ‘test_table’, ‘test_table_snapshot'




c、bin/hbase  -snapshot test_table_snapshot -copy-to hdfs://yourdestination:9100/hbase -mappers 16

Execute the following commands in the target cluster:

a、disable ‘test_table'




b、restore_snapshot ‘test_table_snapshot’




c、enable 'test_table'

3. Turn on REPLICATION of the original table Original cluster execution:

alter ‘test_table’,{NAME=>’f’,REPLICATION_SCOP=>’1’}

4, through the Export tool to snapshot and open replication between the data imported to the slave table; in the original cluster to execute the following instructions:

bin/hbase  test_table hdfs://yourdestination:9100/import/test_table 1 1445961600000 1446048000000

1:Specifies the number of versions, consistent with the line 1445875200000 1445961600000: Specifies the starting timestamp Execute the following command on the target cluster:

bin/hbase  test_table hdfs://yourdestination:9100/import/test_table

Note: 1, the path hdfs://yourdestination:9100/, the port is hdfs external service port, in the configuration, yourdestination is the host name of the active namenode in your destination cluster