WorkLog Frontpage Log in / Register
High-Level Description | Task Dependencies | High-Level Specification | Low-Level Design | File Attachments | User Comments | Time Estimates | Funding and Votes | Progress Reports

 Faster UNIQUE key generation with ALTER TABLE
Title
Task ID231
Queue
Version N/A
Status
PriorityN/A
Copies toSergei

Created byMonty27 Jul 2011Done
Supervisor N/A  
Lead Architect    
Architecture Review  
Implementor  
Code Review  
QA  
Documentation  
 High-Level Description
Faster UNIQUE key generation with ALTER TABLE

Generate also unique keys by sorting (as we do with non unique keys) to speed up
alter tables with unique keys significantly.

This worklog is for doing this for MyISAM and Aria tables. 
 Task Dependencies
Others waiting for Task 231Task 231 is waiting forGraph
 
 High-Level Specification
Currently ALTER TABLE works the following way:

- Create new temporary table
- Disable all non unique keys
- Copy rows, abort if we get a duplicate key
- Enable all the disabled keys (usually with repair by sort)
- Delete original table
- Rename temporary table to original name

The idea would be to disable all keys and when enabling keys do the following
when we encounter a duplicate key:

- If we are not using IGNORE, abort and delete the temporary table
- In case of IGNORE, delete the last found row. For engines that doesn't support
deleting existing rows during recreate index, use old ways of creating unique
keys one by one when using ALTER TABLE IGNORE.


Incompatibility:

- If we use repair to create unique keys and there is a duplicate key and IGNORE
is not used, we can't anymore report on which row the error happened. Instead we
have to just report the duplicated key value.
 Low-Level Design
This changes should be done in MyISAM and Aria:

- Extend the handler call 'start_bulk_insert' with a flag that tells that it's
called from ALTER TABLE (to signal that if something goes wrong we will just
drop the table or drop the conflicting rows).
- Change start_bulk_insert() to disable all index instead of calling
...disable_non_unique_index().
- Extend end_bulk_insert() to gracefully handle duplicate keys.
- Change alter table to check how many rows where deleted so that we can report
  this to the client.



 File Attachments
 NameTypeSizeByDate
 User Comments
 Time Estimates
NameHours WorkedLast Updated
Total0 
 Hrs WorkedProgressCurrentOriginal
This Task03030
Total03030
 
 Funding and Votes
Votes: 1: 100%
 Change vote: Useless    Nice to have    Important    Very important    

Funding: 1 offers, total 500 Euro
 Progress Reports
(Monty - Tue, 18 Sep 2012, 19:07
    
High-Level Specification modified.
--- /tmp/wklog.231.old.32646	2012-09-18 19:07:10.000000000 +0000
+++ /tmp/wklog.231.new.32646	2012-09-18 19:07:10.000000000 +0000
@@ -11,7 +11,10 @@
 when we encounter a duplicate key:
 
 - If we are not using IGNORE, abort and delete the temporary table
-- In case of IGNORE, delete the last found row.
+- In case of IGNORE, delete the last found row. For engines that doesn't support
+deleting existing rows during recreate index, use old ways of creating unique
+keys one by one when using ALTER TABLE IGNORE.
+
 
 Incompatibility:
 

(Monty - Sat, 30 Jul 2011, 06:13
    
High-Level Specification modified.
--- /tmp/wklog.231.old.23850	2011-07-30 06:13:38.000000000 +0000
+++ /tmp/wklog.231.new.23850	2011-07-30 06:13:38.000000000 +0000
@@ -13,4 +13,10 @@
 - If we are not using IGNORE, abort and delete the temporary table
 - In case of IGNORE, delete the last found row.
 
+Incompatibility:
+
+- If we use repair to create unique keys and there is a duplicate key and IGNORE
+is not used, we can't anymore report on which row the error happened. Instead we
+have to just report the duplicated key value.
+
 

(Sergei - Wed, 27 Jul 2011, 20:48
    
Observers changed: Sergei

(Monty - Wed, 27 Jul 2011, 17:05
    
Low Level Design modified.
--- /tmp/wklog.231.old.697	2011-07-27 17:05:04.000000000 +0000
+++ /tmp/wklog.231.new.697	2011-07-27 17:05:04.000000000 +0000
@@ -1,2 +1,15 @@
+This changes should be done in MyISAM and Aria:
+
+- Extend the handler call 'start_bulk_insert' with a flag that tells that it's
+called from ALTER TABLE (to signal that if something goes wrong we will just
+drop the table or drop the conflicting rows).
+- Change start_bulk_insert() to disable all index instead of calling
+...disable_non_unique_index().
+- Extend end_bulk_insert() to gracefully handle duplicate keys.
+- Change alter table to check how many rows where deleted so that we can report
+  this to the client.
+
+
+
 
 

(Monty - Wed, 27 Jul 2011, 16:57
    
High-Level Specification modified.
--- /tmp/wklog.231.old.32745	2011-07-27 16:57:27.000000000 +0000
+++ /tmp/wklog.231.new.32745	2011-07-27 16:57:27.000000000 +0000
@@ -1,2 +1,16 @@
+Currently ALTER TABLE works the following way:
+
+- Create new temporary table
+- Disable all non unique keys
+- Copy rows, abort if we get a duplicate key
+- Enable all the disabled keys (usually with repair by sort)
+- Delete original table
+- Rename temporary table to original name
+
+The idea would be to disable all keys and when enabling keys do the following
+when we encounter a duplicate key:
+
+- If we are not using IGNORE, abort and delete the temporary table
+- In case of IGNORE, delete the last found row.
 
 


Report Generator:
 
Saved Reports:

WorkLog v4.0.0
  © 2010  Sergei Golubchik and Monty Program AB
  © 2004  Andrew Sweger <yDNA@perlocity.org> and Addnorya
  © 2003  Matt Wagner <matt@mysql.com> and MySQL AB