WorkLog Frontpage Log in / Register
High-Level Description | Task Dependencies | High-Level Specification | Low-Level Design | File Attachments | User Comments | Time Estimates | Funding and Votes | Progress Reports

 Multi-source replication
Title
Task ID201
Queue
Version N/A
Status
PriorityN/A
Copies toSergei

Created byKnielsen23 Jun 2011Done
Supervisor N/A  
Lead Architect    
Architecture Review  
Implementor  
Code Review  
QA  
Documentation  
 High-Level Description
Moved to JIRA:
https://mariadb.atlassian.net/browse/MDEV-253
 Task Dependencies
Others waiting for Task 201Task 201 is waiting forGraph
 
 High-Level Specification
Moved to JIRA:
https://mariadb.atlassian.net/browse/MDEV-253
 Low-Level Design
Moved to JIRA:
https://mariadb.atlassian.net/browse/MDEV-253
 File Attachments
 NameTypeSizeByDate
 User Comments
 Time Estimates
NameHours WorkedLast Updated
Knielsen605 Jul 2011
Total6 
 Hrs WorkedProgressCurrentOriginal
This Task6344344
Total6344344
 
 Funding and Votes
Votes: 2: 100%
 Change vote: Useless    Nice to have    Important    Very important    

Funding: 0 offers, total 0 Euro
 Progress Reports
(Rasmus - Fri, 04 May 2012, 10:51
    
High-Level Specification modified.
--- /tmp/wklog.201.old.24189	2012-05-04 10:51:06.000000000 +0000
+++ /tmp/wklog.201.new.24189	2012-05-04 10:51:06.000000000 +0000
@@ -1,72 +1,3 @@
-Here is a preliminary list of things that need to be changed/extended to
-handle multi-source:
-
-
-File names
-----------
-
-These files used to store replication state must be extended to include the
-replicator instance name. For "default", as a special case, the old name
-should be used for backwards compatibility:
-
-    HOST-relay-bin.XXXXXX
-    HOST-relay-bin.index
-    relay-log.info
-    master.info
-
-SQL statements
---------------
-
-These statements need to be extended to take an optional replicator instance
-name. If omitted, "default" is used:
-
-    CHANGE MASTER TO
-    LOAD DATA FROM MASTER
-    LOAD TABLE xxx FROM MASTER
-    MASTER_POS_WAIT(file, position, timeout)
-    RESET SLAVE
-    SHOW SLAVE STATUS
-    START SLAVE
-    START SLAVE UNTIL ...
-    STOP SLAVE
-
-System variables
-----------------
-
-These system variables need to handle multiple replicator instances. Some can
-remain global, and apply to all instances. Some will need to be per-instance,
-and can probably use structured system variables, same was as for multiple key
-caches:
-
-    SQL_SLAVE_SKIP_COUNTER
-    log_slave_updates
-    master_connect_retry
-    master_info_file
-    master_retry_count
-    max_relay_log_size
-    read_only
-    relay_log
-    relay_log_index
-    relay_log_info_file
-    relay_log_purge
-    relay_log_space_limit
-    replicate_do_db
-    replicate_do_table
-    replicate_ignore_db
-    replicate_ignore_table
-    replicate_rewrite_db
-    replicate_same_server_id
-    replicate_wild_do_table
-    replicate_wild_ignore_table
-    report_host
-    report_port
-    skip_slave_start
-    slave_compressed_protocol
-    slave_load_tmpdir
-    slave_net_timeout
-    slave_skip_errors
-
-The deprecated options master_host, master_port, master_user, master_password,
-and master_ssl_* should not be supported for multi source replication.
-
+Moved to JIRA:
+https://mariadb.atlassian.net/browse/MDEV-253
 

(Rasmus - Fri, 04 May 2012, 10:50
    
Low Level Design modified.
--- /tmp/wklog.201.old.24184	2012-05-04 10:50:55.000000000 +0000
+++ /tmp/wklog.201.new.24184	2012-05-04 10:50:55.000000000 +0000
@@ -1,13 +1,3 @@
-A main part of this worklog will be to modify the server code so that all
-slave objects and data can have multiple instances, not use global variables,
-etc. And so that all user-visible interfaces (SQL statements, system
-variables, status variables, files) are extended to support multiple
-replicator instances.
-
-Another major part is to carefully develop the feature and not the least test
-it for full backwards compatibility in the case where only a single, default
-master is used. If this is not done correctly, a _lot_ of users will get
-problems when slaves are upgraded and their monitoring scripts, management
-scripts, or replication state breaks.
-
+Moved to JIRA:
+https://mariadb.atlassian.net/browse/MDEV-253
 

(Rasmus - Fri, 04 May 2012, 10:49
    
High Level Description modified.
--- /tmp/wklog.201.old.24172	2012-05-04 10:49:45.000000000 +0000
+++ /tmp/wklog.201.new.24172	2012-05-04 10:49:45.000000000 +0000
@@ -1,3 +1,3 @@
 Moved to JIRA:
-https://mariadb.atlassian.net/browse/MDEV-136 
+https://mariadb.atlassian.net/browse/MDEV-253
 

(Rasmus - Fri, 04 May 2012, 10:49
    
High Level Description modified.
--- /tmp/wklog.201.old.24160	2012-05-04 10:49:12.000000000 +0000
+++ /tmp/wklog.201.new.24160	2012-05-04 10:49:12.000000000 +0000
@@ -1,38 +1,3 @@
-This worklog is about implementing a way for a MariaDB replication slave to
-replicate from more than one master simultaneously.
-
-Each master is handled by a specific replicator instance in the slave
-server. Each replicator instance consists of separate I/O thread, SQL thread,
-and associated state and configuration. In efffect, several replication slaves
-are running at the same time, each replicating from a separate master, but all
-replicating into a common data store (typically, but not necessarily to
-separate databases/tables).
-
-A replicator instance is identified with a user-chosen name used in
-replication SQL statements such as CHANGE MASTER TO ...
-This name is also included in file names to distinguish the files that keep
-the replication state (relay logs, master.info, relay-log.info).  This way,
-each separate instance can be configured separately, but otherwise the same
-way as existing single-source replication.
-
-In order to remain backwards-compatible with existing third-party scripts
-etc., the replicator instance name is made optional in all existing
-replication statements. If it is omitted, the name "default" is used, and for
-this particular name, master.info and the other files retain their old names
-to allow seamless upgrades of slaves.
-
-In this worklog, there is no extra conflict resolution proposed. The effect of
-updates from one master on replication from another master will be the same as
-the effect of direct user SQL queries on the slave server, ie. it is the
-responsibility of the user/DBA to ensure no conflicts occur. If a conflict
-causes replication of an event to fail (with duplicate key violation eg.), the
-corresponding slave SQL thread will stop, requiring manual intervention to fix
-and start.
-
-An easy and typical way to avoid conflicts would be to eg. to use separate
-databases for each master->slave replication channel. RBR idempotent slave
-application can also be used to help resolve conflicts, for example.
-
-See also MySQL WL#1697: http://forge.mysql.com/worklog/task.php?id=1697
-
+Moved to JIRA:
+https://mariadb.atlassian.net/browse/MDEV-136 
 

(Knielsen - Tue, 05 Jul 2011, 11:37
    
Background research, write up worklog description.
Worked 6 hours and estimate 344 hours remain (original estimate increased by 350 hours).

(Sergei - Thu, 23 Jun 2011, 12:57
    
Observers changed: Sergei

(Knielsen - Thu, 23 Jun 2011, 12:51
    
Low Level Design modified.
--- /tmp/wklog.201.old.28899	2011-06-23 12:51:37.000000000 +0000
+++ /tmp/wklog.201.new.28899	2011-06-23 12:51:37.000000000 +0000
@@ -1,2 +1,13 @@
+A main part of this worklog will be to modify the server code so that all
+slave objects and data can have multiple instances, not use global variables,
+etc. And so that all user-visible interfaces (SQL statements, system
+variables, status variables, files) are extended to support multiple
+replicator instances.
+
+Another major part is to carefully develop the feature and not the least test
+it for full backwards compatibility in the case where only a single, default
+master is used. If this is not done correctly, a _lot_ of users will get
+problems when slaves are upgraded and their monitoring scripts, management
+scripts, or replication state breaks.
 
 

(Knielsen - Thu, 23 Jun 2011, 12:51
    
High-Level Specification modified.
--- /tmp/wklog.201.old.28894	2011-06-23 12:51:23.000000000 +0000
+++ /tmp/wklog.201.new.28894	2011-06-23 12:51:23.000000000 +0000
@@ -1,2 +1,72 @@
+Here is a preliminary list of things that need to be changed/extended to
+handle multi-source:
+
+
+File names
+----------
+
+These files used to store replication state must be extended to include the
+replicator instance name. For "default", as a special case, the old name
+should be used for backwards compatibility:
+
+    HOST-relay-bin.XXXXXX
+    HOST-relay-bin.index
+    relay-log.info
+    master.info
+
+SQL statements
+--------------
+
+These statements need to be extended to take an optional replicator instance
+name. If omitted, "default" is used:
+
+    CHANGE MASTER TO
+    LOAD DATA FROM MASTER
+    LOAD TABLE xxx FROM MASTER
+    MASTER_POS_WAIT(file, position, timeout)
+    RESET SLAVE
+    SHOW SLAVE STATUS
+    START SLAVE
+    START SLAVE UNTIL ...
+    STOP SLAVE
+
+System variables
+----------------
+
+These system variables need to handle multiple replicator instances. Some can
+remain global, and apply to all instances. Some will need to be per-instance,
+and can probably use structured system variables, same was as for multiple key
+caches:
+
+    SQL_SLAVE_SKIP_COUNTER
+    log_slave_updates
+    master_connect_retry
+    master_info_file
+    master_retry_count
+    max_relay_log_size
+    read_only
+    relay_log
+    relay_log_index
+    relay_log_info_file
+    relay_log_purge
+    relay_log_space_limit
+    replicate_do_db
+    replicate_do_table
+    replicate_ignore_db
+    replicate_ignore_table
+    replicate_rewrite_db
+    replicate_same_server_id
+    replicate_wild_do_table
+    replicate_wild_ignore_table
+    report_host
+    report_port
+    skip_slave_start
+    slave_compressed_protocol
+    slave_load_tmpdir
+    slave_net_timeout
+    slave_skip_errors
+
+The deprecated options master_host, master_port, master_user, master_password,
+and master_ssl_* should not be supported for multi source replication.
 
 


Report Generator:
 
Saved Reports:

WorkLog v4.0.0
  © 2010  Sergei Golubchik and Monty Program AB
  © 2004  Andrew Sweger <yDNA@perlocity.org> and Addnorya
  © 2003  Matt Wagner <matt@mysql.com> and MySQL AB