mysql - How can I index these queries?

Question

I am a little confused about the indexing I am going to do.

First, I am using a 4-column index, like this:

Index Name - advanced_query

Columns will be used in the index - title, category 1, category 2, category 3

The Indexing Code

ALTER TABLE table_name ADD INDEX advanced_query (`title`, `cat_1`, `cat_2`, `cat_3`, `date_posted`)

Okay, so this is how (from what I understand) it will work:

• a query of title will use the index.
• a query of cat_1 will use the index.
• a query of cat_2 will use the index.
• a query of cat_3 will NOT use the index. So I will create a different index for it.
• a query of title, cat_1 will use the index.
• a query of title, cat_1, cat_2 will use the index.
• a query of title, cat_1, cat_2, cat_3 will use the index.
• a query of title, cat_1, cat_3 will use the index.
• a query of title, cat_2 will use the index.
• a query of title, cat_2, cat_3 will use the index.
• a query of title, cat_3 will use the index.
• a query of cat_1, cat_2 will use the index.
• a query of cat_1, cat_2, cat_3 will use the index.
• a query of cat_1, cat_2 will use the index.
• a query of cat_1, cat_3 will use the index.

TL;DR

So in this index, only a query of cat_3 will not benefit from it, right? Thanks!

Q/A

What query am I doing ? searching a post (it's title and 3 different categories)

What is the size of table ? Less than 2 thousand rows

Structure of the table ?

CREATE TABLE `post_lists` (
 `id` int(100) NOT NULL AUTO_INCREMENT,
 `users_id` varchar(100) NOT NULL,
 `code` varchar(255) NOT NULL,
 `date_posted` datetime NOT NULL,
 `date_updated` datetime NOT NULL,
 `title` varchar(255) NOT NULL,
 `cat_1` varchar(255) NOT NULL,
 `cat_3_code` varchar(255) NOT NULL,
 `details` varchar(10000) NOT NULL,
 `cat_2` varchar(255) NOT NULL,
 `cat_3` varchar(255) NOT NULL,
 UNIQUE KEY `id` (`id`),
 KEY `date_posted` (`date_posted`),
 KEY `code` (`urlcode`),
 KEY `users_id_date_posted` (`users_id`,`date_posted`),
 KEY `title_date_posted` (`title`,`date_posted`),
 KEY `cat_1_date_posted` (`cat_1`,`date_posted`)
) ENGINE=InnoDB AUTO_INCREMENT=37 DEFAULT CHARSET=latin1

How many times will this table use ? Most of the time. This is the advanced search function so not just frequently as the basic search is.

This is how I actually will use the index.

Example Table

title | cat_1 | cat_2 | cat_3 | date_posted

My queries are simple:

1. title

   SELECT * FROM tbl_name WHERE title LIKE %title% ORDER BY date_posted DESC

2. title + cat_1

   SELECT * FROM tbl_name WHERE title LIKE %title% AND cat_1 = 'cat_1' ORDER BY date_posted DESC

3. title + cat_1 + cat_2

   SELECT * FROM tbl_name WHERE title LIKE %title% AND cat_1 = 'cat_1' AND cat_2 = 'cat_2' ORDER BY date_posted DESC

4. title + cat_1 + cat_2 + cat_3

   SELECT * FROM tbl_name WHERE title LIKE %title% AND cat_1 = 'cat_1' AND cat_2 = 'cat_2' AND cat_3 = 'cat_3' ORDER BY date_posted DESC

5. title + cat_1 + cat_3

   SELECT * FROM tbl_name WHERE title LIKE %title% AND cat_1 = 'cat_1' and cat_3 = 'cat_3' ORDER BY date_posted DESC

6. title + cat_2

   SELECT * FROM tbl_name WHERE title LIKE %title% AND cat_2 = 'cat_2' ORDER BY date_posted DESC

7. title + cat_2 + cat_3

   SELECT * FROM tbl_name WHERE title LIKE %title% AND cat_2 = 'cat_2' AND cat_3 = 'cat_3' ORDER BY date_posted DESC

8. title + cat_3

   SELECT * FROM tbl_name WHERE title LIKE %title% AND cat_3 = 'cat_3' ORDER BY date_posted DESC

9. cat_1

   SELECT * FROM tbl_name WHERE cat_1 = 'cat_1' ORDER BY date_posted DESC

10. cat_1 + cat_2

    SELECT * FROM tbl_name WHERE cat_1 = 'cat_1' AND cat_2 = 'cat_2' ORDER BY date_posted DESC

11. cat_1 + cat_2 + cat_3

    SELECT * FROM tbl_name WHERE cat_1 = 'cat_1' AND cat_2 = 'cat_2' AND cat_3 = 'cat_3' ORDER BY date_posted DESC

12. cat_1 + cat_3

    SELECT * FROM tbl_name WHERE cat_1 = 'cat_1' AND cat_3 = 'cat_3' ORDER BY date_posted DESC

13. cat_2

    SELECT * FROM tbl_name WHERE cat_2 = 'cat_2' ORDER BY date_posted DESC

14. cat_2 + cat_3

    SELECT * FROM tbl_name WHERE cat_2 = 'cat_2' ORDER BY date_posted DESC

15. cat_3

    SELECT * FROM tbl_name WHERE cat_3 = 'cat_3' ORDER BY date_posted DESC

How can I query this?

Edit

Hi, I read and searched about Full Text Search, and I am thinking to use it (in basic search) instead of LIKE %wildcard% and applying it to title and details, my problem is I want them to sort ORDER BY date_posted DESC, so should I add date_posted in Full Text Search or create a separate index?

Answer 1

I think an "answer" to your question is a bit complex and your supposition on index use are not always correct.

The shorter answer would be: "It depends".

Actually the use of an index depends by several factor: number of records in table, index structure, field requested, condition in query, statistics.

1) Number of records: if it is small, maybe db engine decide not to use the index (especially if you write SELECT * of SELECT --several columns in table not in index --).

The index could be used (not considering WHERE condition too) if you SELECT only some of or all the columns in index.

2) index structure: as you pointed, it's relevant. Morevore there are two different main ways an index can be "used": scan and seek. Seek is the most efficient. In most cases you have a seek if you look for the columns in the index in the same order you wrote them: eg. SELECT TITLE FROM YOUR TABLE WHERE TITLE LIKE 'ABC%'). Note: if you wrote LIKE '%ABC%' it can't do a seek, but a scan. (A scan mean that db have to look for the whole index, from the beginning to the end, while with a seek he goes directly to the relevant pages, as you will do looking for a phone number of a person in a phonebook using lastname).

3) Field requested: you should consider that if you write SELECT * (as I pointed above, db engine could decide to use anyway a full table scan)

4) Condition in query.

5) Statistics: db engine write statistics on data and index (number of record, structure,etc). If they are not updated, it's possibile it use or don't use index in "erroneus" way.

----- Updated : simple (not exhaustive ... ) demo

Actually (with this small data, I had to comment your KEY 'title_date_posted' to make it use in some cases the "advanced_query" index: otherwise it seems to try to use that; as I told you, db engine makes an internal decisions what index to use).

Test done on rextester.com:

##DROP TABLE post_lists;

CREATE TABLE `post_lists` (
 `id` int(100) NOT NULL AUTO_INCREMENT,
 `users_id` varchar(100) NOT NULL,
 `code` varchar(255) NOT NULL,
 `date_posted` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
 `date_updated` datetime NOT NULL  DEFAULT CURRENT_TIMESTAMP,
 `title` varchar(255) NOT NULL,
 `cat_1` varchar(255) NOT NULL,
 `cat_3_code` varchar(255) NOT NULL,
 `details` varchar(10000) NULL,
 `cat_2` varchar(255) NOT NULL,
 `cat_3` varchar(255) NOT NULL,
 UNIQUE KEY `id` (`id`)
    , KEY `date_posted` (`date_posted`)
    , KEY `code` (`code`)
    , KEY `users_id_date_posted` (`users_id`,`date_posted`)
    ##, KEY `title_date_posted` (`title`,`date_posted`)
    , KEY `cat_1_date_posted` (`cat_1`,`date_posted`)
)   DEFAULT CHARSET=latin1;

ALTER TABLE post_lists ADD INDEX advanced_query (`title`, `cat_1`, `cat_2`, `cat_3`, `date_posted`);

INSERT INTO post_lists (users_id, code, title, cat_1, cat_3_code, details, cat_2, cat_3) VALUES ('123', 'ABC', 'TITLE1', '001','C3','blah blah blah', '002', '003');
INSERT INTO post_lists (users_id, code, title, cat_1, cat_3_code, details, cat_2, cat_3) VALUES ('456', 'ABC', 'TITLE2', '002','C32','blah blah blah', '0021', '0031');

SELECT * FROM post_lists;

EXPLAIN SELECT * FROM post_lists WHERE title = 'TITLE1'; 
EXPLAIN SELECT title FROM post_lists  WHERE title = 'TITLE1'; 
EXPLAIN SELECT title, cat_1, cat_3, code FROM post_lists  WHERE title = 'TITLE1'; 
EXPLAIN SELECT title, cat_1, cat_3 FROM post_lists  WHERE title = 'TITLE1'; 

DROP TABLE post_lists;

Output:

    +----+----+----------+------+---------------------+---------------------+--------+-------+------------+----------------+-------+-------+
|    | id | users_id | code |     date_posted     |    date_updated     | title  | cat_1 | cat_3_code |    details     | cat_2 | cat_3 |
+----+----+----------+------+---------------------+---------------------+--------+-------+------------+----------------+-------+-------+
|  1 |  1 |      123 | ABC  | 27.06.2017 11:02:16 | 27.06.2017 11:02:16 | TITLE1 |   001 | C3         | blah blah blah |   002 |   003 |
|  2 |  2 |      456 | ABC  | 27.06.2017 11:02:16 | 27.06.2017 11:02:16 | TITLE2 |   002 | C32        | blah blah blah |  0021 |  0031 |
+----+----+----------+------+---------------------+---------------------+--------+-------+------------+----------------+-------+-------+

+----+----+-------------+------------+------------+------+----------------+----------------+---------+-------+------+----------+-------+
|    | id | select_type |   table    | partitions | type | possible_keys  |      key       | key_len |  ref  | rows | filtered | Extra |
+----+----+-------------+------------+------------+------+----------------+----------------+---------+-------+------+----------+-------+
|  1 |  1 | SIMPLE      | post_lists | NULL       | ref  | advanced_query | advanced_query |     257 | const |    1 |      100 | NULL  |
+----+----+-------------+------------+------------+------+----------------+----------------+---------+-------+------+----------+-------+



+----+----+-------------+------------+------------+------+----------------+----------------+---------+-------+------+----------+-------------+
|    | id | select_type |   table    | partitions | type | possible_keys  |      key       | key_len |  ref  | rows | filtered |    Extra    |
+----+----+-------------+------------+------------+------+----------------+----------------+---------+-------+------+----------+-------------+
|  1 |  1 | SIMPLE      | post_lists | NULL       | ref  | advanced_query | advanced_query |     257 | const |    1 |      100 | Using index |
+----+----+-------------+------------+------------+------+----------------+----------------+---------+-------+------+----------+-------------+

+----+----+-------------+------------+------------+------+----------------+----------------+---------+-------+------+----------+-------+
|    | id | select_type |   table    | partitions | type | possible_keys  |      key       | key_len |  ref  | rows | filtered | Extra |
+----+----+-------------+------------+------------+------+----------------+----------------+---------+-------+------+----------+-------+
|  1 |  1 | SIMPLE      | post_lists | NULL       | ref  | advanced_query | advanced_query |     257 | const |    1 |      100 | NULL  |
+----+----+-------------+------------+------------+------+----------------+----------------+---------+-------+------+----------+-------+


+----+----+-------------+------------+------------+------+----------------+----------------+---------+-------+------+----------+-------------+
|    | id | select_type |   table    | partitions | type | possible_keys  |      key       | key_len |  ref  | rows | filtered |    Extra    |
+----+----+-------------+------------+------------+------+----------------+----------------+---------+-------+------+----------+-------------+
|  1 |  1 | SIMPLE      | post_lists | NULL       | ref  | advanced_query | advanced_query |     257 | const |    1 |      100 | Using index |
+----+----+-------------+------------+------------+------+----------------+----------------+---------+-------+------+----------+-------------+