How do I rewrite my MySQL update statement to eliminate "Truncated incorrect INTEGER value" warnings?

Question

I'm using MySQL 5.5.37. I want to eliminate the warnings from my update statement, which are shown below ...

update resource r 
    set grade_id = convert(substring_index(substring_index(
                   r.description, 'Grade ', -1), ' ', 1), unsigned integer) 
    where r.description like '% Grade%' 
      and CONVERT(SUBSTRING_INDEX(SUBSTRING_INDEX(
                   r.description, 'Grade ', -1), ' ' ,1),UNSIGNED) > 0;

Query OK, 0 rows affected, 7 warnings (0.02 sec)
Rows matched: 1333  Changed: 0  Warnings: 7

mysql> show warnings;
+---------+------+--------------------------------------------------+
| Level   | Code | Message                                          |
+---------+------+--------------------------------------------------+
| Warning | 1292 | Truncated incorrect INTEGER value: ''            |
| Warning | 1292 | Truncated incorrect INTEGER value: ''            |
| Warning | 1292 | Truncated incorrect INTEGER value: ''            |
| Warning | 1292 | Truncated incorrect INTEGER value: 'MyCo'        |
| Warning | 1292 | Truncated incorrect INTEGER value: 'MyCo'        |
| Warning | 1292 | Truncated incorrect INTEGER value: 'MyCo'        |
| Warning | 1292 | Truncated incorrect INTEGER value: 'MyCo'        |
+---------+------+--------------------------------------------------+

What I do not understand is how I can rewrite my query differently given that none of the values being updated match what the warnings are complaining about. BElow is my query where I list the distinct values that are being updated ...

mysql> select distinct substring_index(substring_index(
              r.description, 'Grade ', -1), ' ', 1)
          from resource r
          where r.description like '% Grade%'
            and CONVERT(SUBSTRING_INDEX(SUBSTRING_INDEX(
              r.description, 'Grade ',-1),' ',1),UNSIGNED) > 0;
+-----------------------------------------------------------------------+
| substring_index(substring_index(r.description, 'Grade ', -1), ' ', 1) |
+-----------------------------------------------------------------------+
| 7                                                                     |
| 8                                                                     |
| 11                                                                    |
| 9                                                                     |
| 12                                                                    |
| 10                                                                    |
| 6                                                                     |
+-----------------------------------------------------------------------+

How do I rewrite my update statement so that it updates the same values without tryihng to truncate incorrect integers?

Solution 1

As far as my knowledge concern you get the warning due to WHERE clause condition

CONVERT(SUBSTRING_INDEX(SUBSTRING_INDEX(r.description, 'Grade ',-1),' ',1),UNSIGNED) > 0

As clearly mention in warning statement Truncated due to incorrect INTEGER value for '' and MyCo value.

| Level   | Code | Message                                          |
+---------+------+--------------------------------------------------+
| Warning | 1292 | Truncated incorrect INTEGER value: ''            |
| Warning | 1292 | Truncated incorrect INTEGER value: ''            |
| Warning | 1292 | Truncated incorrect INTEGER value: ''            |
| Warning | 1292 | Truncated incorrect INTEGER value: 'MyCo'        |
| Warning | 1292 | Truncated incorrect INTEGER value: 'MyCo'        |
| Warning | 1292 | Truncated incorrect INTEGER value: 'MyCo'        |
| Warning | 1292 | Truncated incorrect INTEGER value: 'MyCo' 

Which indicate that you did not have any specific pattern for value in description column of resource table.

Like below

description
Foo Grade 100 Bar
Foo Grade 99 Grade 
Foo Grade 98 Grade MyCO

As shown in above code in second & third row you have multiple Grade string in record. Which indirectly affect SUBSTRING_INDEX(SUBSTRING_INDEX(r.description, 'Grade ',-1),' ',1), line. So what my suggestion is please make sure that all record inserted properly. If all record are inserted properly and Check that they contains multiple types of pattern for descripton column.If contains multiple pattern then you have to rewrite this query using UNION. Single query for one pattern and another query for another pattern.

And just try below updated query.

   update resource r 
    set grade_id = convert(substring_index(substring_index(
                   r.description, 'Grade ', -1), ' ', 1), unsigned integer) 
    where r.description like '% Grade%' 
    and substring_index(substring_index(r.description, 'Grade ', -1), ' ', 1) REGEXP '[0-9]+';

Hope this explanation helps you.

Solution 2

You get the warnings because of the WHERE clause, not the SELECT clause. Since invalid (non numeric) values are converted to 0 the condition

CONVERT(SUBSTRING_INDEX(SUBSTRING_INDEX(
          r.description, 'Grade ',-1),' ',1),UNSIGNED) > 0

retuns false. Thus these rows do not appear in the result. Remove that condition in you select statement to see which rows cause the warnings:

select distinct 
    substring_index(substring_index(
        r.description, 'Grade ', -1), ' ', 1),
    CONVERT(SUBSTRING_INDEX(SUBSTRING_INDEX(
        r.description, 'Grade ',-1),' ',1),UNSIGNED),
        r.description
from resource r
where r.description like '% Grade%';

Example: http://rextester.com/TVHN10678

Since i don't know your data, i can't help to fix it.

Solution 3

can you use trim() before convert to remove any whitespaces that may have crept in? those are also the reason for this warning

Solution 4

You can try to use regular expression, something like:

where r.description like '% Grade%' and SUBSTRING_INDEX(SUBSTRING_INDEX( r.description, 'Grade ', -1), ' ' ,1) REGEXP '^[0-9]+$';

Alternatively if you are not afraid of stepping into gray area: you can also try the same query without CONVERT at all - surprisingly it may just work:

where r.description like '% Grade%' and SUBSTRING_INDEX(SUBSTRING_INDEX( r.description, 'Grade ', -1), ' ' ,1) > 0;

(again I am not sure if this is documented behavior, thus - 'gray area').

Solution 5

Somewhere in your resource.description column you have either '' or MyCo as per your warnings output. There is non-numeric data being passed to the convert function, which is where the warnings are coming from.

I don't know what your raw data looks like, but if you run the below query, it should identify any rows where resource.description is not in the correct format.

select distinct substring_index(substring_index(r.description, 'Grade ', -1), ' ', 1)
from resource r 
order by r.description

You can try running an EXPLAIN on your query as well. This may give you more insight. I don't know the internals of MySQL, but perhaps the convert function is somehow looking at all rows, despite having a where clause.

View more solutions

Comments

[Rick James]

Please show us a line with MyCo in it. From that, we can work forward.

[Paul Spiegel]

Hmm.. Your example looks very similar to this rexester demo. Was that just a coincidence?

[Rick James]

Needs to be "anchored": REGEXP '^[0-9]+$'