row0ftsort.cc File Reference

#include "dict0dict.h"
#include "row0merge.h"
#include "pars0pars.h"
#include "row0ftsort.h"
#include "row0row.h"
#include "btr0cur.h"

Include dependency graph for row0ftsort.cc:

Go to the source code of this file.

Macros
#define	ROW_MERGE_READ_GET_NEXT(N)

Functions
UNIV_INTERN dict_index_t *	row_merge_create_fts_sort_index (dict_index_t *index, const dict_table_t *table, ibool *opt_doc_id_size)
UNIV_INTERN ibool	row_fts_psort_info_init (trx_t *trx, row_merge_dup_t *dup, const dict_table_t *new_table, ibool opt_doc_id_size, fts_psort_t **psort, fts_psort_t **merge)
UNIV_INTERN void	row_fts_psort_info_destroy (fts_psort_t *psort_info, fts_psort_t *merge_info)
UNIV_INTERN void	row_fts_free_pll_merge_buf (fts_psort_t *psort_info)
UNIV_INTERN os_thread_ret_t	fts_parallel_tokenization (void *arg)
UNIV_INTERN void	row_fts_start_psort (fts_psort_t *psort_info)
UNIV_INTERN os_thread_ret_t	fts_parallel_merge (void *arg)
UNIV_INTERN void	row_fts_start_parallel_merge (fts_psort_t *merge_info)

Variables
UNIV_INTERN ulong	fts_sort_pll_degree = 2
static que_t **	ins_graph
static que_t fts_tokenizer_word_t *	word
static que_t fts_tokenizer_word_t fts_table_t *	fts_table

Detailed Description

Create Full Text Index with (parallel) merge sort

Created 10/13/2010 Jimmy Yang

Definition in file row0ftsort.cc.

Macro Definition Documentation

#define ROW_MERGE_READ_GET_NEXT

(

N

)

Value:

do {                                                            \
                b[N] = row_merge_read_rec(                              \
                        block[N], buf[N], b[N], index,                  \
                        fd[N], &foffs[N], &mrec[N], offsets[N]);        \
                if (UNIV_UNLIKELY(!b[N])) {                             \
                        if (mrec[N]) {                                  \
                                goto exit;                              \
                        }                                               \
                }                                                       \
        } while (0)

Read the next record to buffer N.

Parameters

N	index into array of merge info structure

Definition at line 36 of file row0ftsort.cc.

Function Documentation

UNIV_INTERN os_thread_ret_t fts_parallel_merge

(

void *

arg

)

Function performs the merge and insertion of the sorted records.

Returns

OS_THREAD_DUMMY_RETURN

Parameters

arg	in: parallel merge info

Definition at line 881 of file row0ftsort.cc.

Here is the call graph for this function:

Here is the caller graph for this function:

UNIV_INTERN os_thread_ret_t fts_parallel_tokenization

(

void *

arg

)

Function performs parallel tokenization of the incoming doc strings. It also performs the initial in memory sort of the parsed records.

Returns

OS_THREAD_DUMMY_RETURN

Parameters

arg	in: psort_info for the thread

Definition at line 553 of file row0ftsort.cc.

Here is the call graph for this function:

Here is the caller graph for this function:

UNIV_INTERN void row_fts_free_pll_merge_buf

(

fts_psort_t *

psort_info

)

Free up merge buffers when merge sort is done

Parameters

psort_info

in: parallel sort info

Definition at line 332 of file row0ftsort.cc.

Here is the call graph for this function:

UNIV_INTERN void row_fts_psort_info_destroy	(	fts_psort_t *	psort_info,
		fts_psort_t *	merge_info
	)

Clean up and deallocate FTS parallel sort structures, and close the merge sort files

Parameters

psort_info	parallel sort info
merge_info	parallel merge info

Definition at line 294 of file row0ftsort.cc.

Here is the call graph for this function:

Here is the caller graph for this function:

UNIV_INTERN ibool row_fts_psort_info_init	(	trx_t *	trx,
		row_merge_dup_t *	dup,
		const dict_table_t *	new_table,
		ibool	opt_doc_id_size,
		fts_psort_t **	psort,
		fts_psort_t **	merge
	)

Initialize FTS parallel sort structures.

Returns

TRUE if all successful

Parameters

trx	in: transaction
dup	in,own: descriptor of FTS index being created
new_table	in: table on which indexes are created
opt_doc_id_size	in: whether to use 4 bytes instead of 8 bytes integer to store Doc ID during sort
psort	out: parallel sort info to be instantiated
merge	out: parallel merge info to be instantiated

Definition at line 171 of file row0ftsort.cc.

Here is the call graph for this function:

UNIV_INTERN void row_fts_start_parallel_merge

(

fts_psort_t *

merge_info

)

Kick off the parallel merge and insert thread

Parameters

merge_info

in: parallel sort info

Definition at line 913 of file row0ftsort.cc.

Here is the call graph for this function:

UNIV_INTERN void row_fts_start_psort

(

fts_psort_t *

psort_info

)

Start the parallel tokenization and parallel merge sort

Parameters

psort_info

parallel sort structure

Definition at line 861 of file row0ftsort.cc.

Here is the call graph for this function:

UNIV_INTERN dict_index_t* row_merge_create_fts_sort_index	(	dict_index_t *	index,
		const dict_table_t *	table,
		ibool *	opt_doc_id_size
	)

Create a temporary "fts sort index" used to merge sort the tokenized doc string. The index has three "fields":

1) Tokenized word, 2) Doc ID (depend on number of records to sort, it can be a 4 bytes or 8 bytes integer value) 3) Word's position in original doc.

Returns

dict_index_t structure for the fts sort index

Parameters

index	in: Original FTS index based on which this sort index is created
table	in: table that FTS index is being created on
opt_doc_id_size	out: whether to use 4 bytes instead of 8 bytes integer to store Doc ID during sort

Definition at line 63 of file row0ftsort.cc.

Here is the call graph for this function:

Variable Documentation

UNIV_INTERN ulong fts_sort_pll_degree = 2

Parallel sort degree

Definition at line 49 of file row0ftsort.cc.

que_t fts_tokenizer_word_t fts_table_t* fts_table

in: fts aux table instance

Definition at line 938 of file row0ftsort.cc.

que_t** ins_graph

< in: transaction in: Insert query graphs

Definition at line 938 of file row0ftsort.cc.

que_t fts_tokenizer_word_t* word

in: sorted and tokenized word

Definition at line 938 of file row0ftsort.cc.