nlpir.native package¶

class nlpir.native.ICTCLAS(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

A dynamic link library native class for Chinese Segmentation

POS_MAP_NUMBER = 4¶

ICT_POS_MAP_FIRST = 1¶

ICT_POS_MAP_SECOND = 0¶

PKU_POS_MAP_SECOND = 2¶

PKU_POS_MAP_FIRST = 3¶

POS_SIZE = 40¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call NLPIR_Init

Parameters:	data_path (str) – encode (int) – license_code (str) –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call NLPIR_Exit

Returns:	exit success or not

paragraph_process(paragraph: str, pos_tagged: int = 1) → str[source]¶

Call NLPIR_ParagraphProcessing

Chinese word segment, segment paragraph to a string

Parameters:	paragraph (str) – the string want to be segmented pos_tagged (int) – show the pos tag or not 1-> True, 0-> False
Returns:	segmented string

paragraph_process_a(paragraph: str, user_dict: bool = True) → Tuple[nlpir.native.ictclas.ResultT, int][source]¶

Call ParagraphProcessingA

Segment paragraph to an Array of ResultT, get more detail info

Parameters:	paragraph (str) – the string want to be segmented user_dict (bool) – use user dictionary or not
Returns:	a result of segment, an array of ResultT and the length of the ResultT

get_paragraph_process_a_word_count(paragraph: str) → int[source]¶

paragraph_process_aw(count: int, result: nlpir.native.ictclas.ResultT) → None[source]¶

file_process(source_filename: str, result_filename: str, pos_tagged: int = 1) → float[source]¶

Call NLPIR_FileProcess

Segment a text file and save to a file.

Parameters:	source_filename (str) – the path of a text file that want to be segmented result_filename (str) – the path to save the result of segmentation pos_tagged (int) – show the pos tag or not 1->True, 0->False

import_user_dict(filename: str, overwrite: bool = False) → int[source]¶

Call NLPIR_ImportUserDict

Import a user dict to the system, the format of the dict file:

word1 pos_tag
word2 pos_tag

If you import a user dict to the system, the user dict will save to the system (in Data directory). You cannot delete the word in the user dict from the system use clean_user_word() or del_usr_word().

TODO add more comment for clean the user dict and add the function to the high-level method

Parameters:	filename (str) – the path of user dict file overwrite (bool) – overwrite the current user dict or not
Returns:	import success or not 1->True 2->False

add_user_word(word: str) → int[source]¶

Call NLPIR_AddUserWord

Add a word to the user dictionary ,example:

单词 词性

or:

单词 (default n)

The added word only add in memory and will not affect the user dict, you can use clean_user_word() or del_usr_word() to delete the word or all the words in memory. If you want to save to the user dict ,use save_the_usr_dic() to save to the Data directory.

Parameters:	word (str) –
Returns:	1,true ; 0,false

clean_user_word() → int[source]¶

Call NLPIR_CleanUserWord

Clean all temporary added user words, more info see add_user_word() TODO figure out the return value :return: 1,true ; 0,false

clean_current_user_word() → int[source]¶

Call NLPIR_CleanCurrentUserWord Clean all Current temporary added user words and restore previous stored data

** Now Only for win and linux x64 **

Returns:	1,true; 2,false

save_the_usr_dic() → int[source]¶

Call NLPIR_SaveTheUsrDic

Save in-memory dict to user dict, more info see add_user_word()

Returns:	1,true; 2,false

del_usr_word(word: str) → int[source]¶

Call NLPIR_DelUsrWord

Delete a word from the user dictionary, more info see add_user_word()

Parameters:	word (str) – the word to delete
Returns:	-1, the word not exist in the user dictionary; else, the handle of the word deleted

get_uni_prob(word) → float[source]¶

Call NLPIR_GetUniProb

Get Unigram Probability

Parameters:	word (str) – input word
Returns:	The unitary probability of a word.

is_word(word: str) → int[source]¶

Call NLPIR_IsWord

Judge whether the word is included in the core dictionary

Parameters:	word (str) – input word
Returns:	1: exists; 0: no exists

is_user_word(word: str, is_ascii: bool = False) → int[source]¶

Call NLPIR_IsUserWord

Judge whether the word is included in the user-defined dictionary

Parameters:	word (str) – input word is_ascii (bool) – is ascii encode or not
Returns:	1: exists; 0: no exists

get_word_pos(word: str) → str[source]¶

Call NLPIR_GetWordPOS

Get the word Part-Of-Speech information

Parameters:	word (str) – input word
Returns:	pos tagging

set_pos_map(pos_map: int) → int[source]¶

Call NLPIR_SetPOSmap

Select which pos map will use:

ICT_POS_MAP_FIRST 计算所一级标注集
ICT_POS_MAP_SECOND 计算所二级标注集
PKU_POS_MAP_SECOND 北大二级标注集
PKU_POS_MAP_FIRST 北大一级标注集

Default is ICT_POS_MAP_SECOND

Parameters:	pos_map (int) –
Returns:	0, failed; else, success

finer_segment(line: str) → str[source]¶

Call NLPIR_FinerSegment

当前的切分结果过大时,如“中华人民共和国”, 需要执行该函数,将切分结果细分为“中华人民共和国”

细分粒度最大为三个汉字,如果不能细分，则返回为空字符串

Parameters:	line (str) – string need to be segmented
Returns:	segmented string, return null string if line cannot be segmented

get_eng_word_origin(word: str) → str[source]¶

Call NLPIR_GetEngWordOrign

获取各类英文单词的原型，考虑了过去分词、单复数等情况:

driven->drive   drives->drive   drove-->drive

Parameters:	word (str) – word to be stemmed
Returns:	the stemmed word

word_freq_stat(text: str, stop_word_remove: bool = True) → str[source]¶

Call NLPIR_WordFreqStat

获取输入文本的词，词性，频统计结果，按照词频大小排序

Parameters:	text (str) – 输入的文本内容 stop_word_remove (bool) – true-去除停用词 false-不去除停用词
Returns:	返回的是词频统计结果形式如下

张华平/nr/10#博士/n/9#分词/n/8

file_word_freq_stat(filename: str, stop_word_remove: bool = True) → str[source]¶

Call NLPIR_FileWordFreqStat

Same as word_freq_stat()

Parameters:	filename (str) – path of text file stop_word_remove (bool) – remove stop words or not
Returns:	same as `word_freq_stat()`

get_last_error_msg() → str[source]¶

Call NLPIR_GetLastErrorMsg

Returns:	error message

tokenizer_for_ir(text: str, fine_segment: bool = False) → str[source]¶

Call NLPIR_Tokenizer4IR

搜索引擎模式，在精确模式的基础上，对长词再次切分，提高召回率，适合用于搜索引擎分词

Parameters:	text (str) – The source paragraph fine_segment (bool) – Need finer segment or not
Returns:

输入：国务院办公厅转发商务部的结果如下：

[
    {
    "begin" : 0,
    "end" : 6,
    "pos" : "nt",
    "text" : "国务院办公厅"
    },
    {
    "begin" : 0,
    "end" : 3,
    "pos" : "",
    "text" : "国务院"
    },
    {
    "begin" : 3,
    "end" : 6,
    "pos" : "",
    "text" : "办公厅"
    },
    {
    "begin" : 6,
    "end" : 8,
    "pos" : "v",
    "text" : "转发"
    },
    {
    "begin" : 8,
    "end" : 11,
    "pos" : "n",
    "text" : "商务部"
    }
]

class nlpir.native.Classifier(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call classifier_init

Parameters:	data_path – encode – license_code –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call classifier_exit

Returns:	exit success or not

get_last_error_msg() → str[source]¶: 对应每个组件获取异常信息的函数

exec_1(data: nlpir.native.classifier.StDoc, out_type: int = 0)[source]¶

Call classifier_exec1

对输入的文章结构进行分类

Parameters:	data – 文章结构 out_type – 输出是否包括置信度, 0 没有置信度 1 有置信度
Returns:	主题类别串各类之间用隔开，类名按照置信度从高到低排序举例：“要闻敏感诉讼”, “要闻 1.00 敏感 0.95 诉讼 0.82”

exec(title: str, content: str, out_type: int)[source]¶

Call classifier_exec

对输入的文章进行分类

Parameters:	title – 文章标题 content – 文章内容 out_type – 输出知否包括置信度,同 `exec_1()`
Returns:	同 `exec_1()`

exec_file(filename: str, out_type: int) → str[source]¶

Call classifier_execFile

Parameters:	filename – 文件名 out_type – 输出是否包括置信度, 0 没有置信度 1 有置信度
Returns:	主题类别串各类之间用隔开，类名按照置信度从高到低排序举例：“要闻敏感诉讼”, “要闻 1.00 敏感 0.95 诉讼 0.82”

detail(class_name: str)[source]¶

Call classifier_detail

对于当前文档，输入类名，取得结果明细

Parameters:	class_name – 结果类名
Returns:	结果明细例如:

RULE3:
SUBRULE1: 内幕 1
SUBRULE2: 股市 1      基金 3    股票 8
SUBRULE3: 书摘 2

set_sim_thresh(sim: float)[source]¶

Call classifier_setsimthresh

设置阈值

Parameters:	sim – 阈值
Returns:

class nlpir.native.Cluster(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

load_mode = 1¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call CLUS_init

Parameters:	data_path – encode – license_code –
Returns:	1 success Other fail

exit_lib() → bool[source]¶

Call CLUS_exit

Returns:	exit success or not

get_last_error_msg() → str[source]¶

Call CLUS_GetLastErrorMsg

Returns:

set_parameter(max_clus: int, max_doc: int) → bool[source]¶

Call CLUS_SetParameter

设置最大类别数以及最大输入文档数,类和类内的文档均已按照重要性和及时性排过序

Parameters:	max_clus – 最大类别数 max_doc – 最大文档数
Returns:	是否成功

add_content(text: str, signature: str) → bool[source]¶

Call CLUS_AddContent

追加内存内容,在进程中此函数可以在打印结果之前执行多次

Parameters:	text – 正文 signature – 唯一标识
Returns:	是否成功

add_file(filename: str)[source]¶

Call CLUS_AddFile

追加文件内容,在进程中此函数可以在打印结果之前执行多次

Parameters:	filename – 正文文件
Returns:	是否成功

get_latest_result(xml_filename: str, result_path: Optional[str] = None) → Tuple[bool, str][source]¶

Call CLUS_GetLatestResult

输出结果到xml文件中

<?xml version="1.0" encoding="gb2312" standalone="yes" ?>
<LJCluster-Result>
<clusnum>2</clusnum>

<clus id="0">
    <feature>奥巴马 竞选 财务部</feature>
    <docs num="6">
       <doc>2</doc>
       <doc>3</doc>
       <doc>35</doc>
       <doc>86</doc>
       <doc>345</doc>
       <doc>975</doc>
    </docs>
</clus>

<clus id="1">
    <feature>林志玲 影视 电影 广告</feature>
    <docs num="4">
       <doc>45</doc>
       <doc>86</doc>
       <doc>135</doc>
       <doc>286</doc>
    </docs>
</clus>
</LJCluster-Result>

Parameters:	xml_filename – 输出文件名 result_path – 输出路径, 按照聚类结果作为不同子目录存储
Returns:	是否成功

get_latest_result_e(result_path: Optional[str] = None) → str[source]¶

Call CLUS_GetLatestResultE

输出xml结果到内存

Parameters:	result_path –
Returns:	xml like `get_latest_result()`

clean_data() → None[source]¶

Call CLUS_CleanData

清空历史数据

Returns:

class nlpir.native.EyeChecker(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

TODO report_type or doc_type

A dynamic link library native class for 09 Eys Checker

DOC_EXTRACT_DELIMITER = '#'¶: 分隔符

DOC_EXTRACT_TYPE_MAX_LENGTH = 600¶

load_mode = 1¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call NERICS_Init

Parameters:	data_path (str) – encode (int) – license_code (str) –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call NERICS_Exit

Returns:	exit success or not

get_last_error_msg() → str[source]¶

Call DE_GetLastErrorMsg

Returns:	error message

import_field_dict(field_dict_file: str, pinyin_abbrev_needed: bool = False, overwrite: bool = True) → int[source]¶

Import field dictionary

Parameters:	field_dict_file – pinyin_abbrev_needed – overwrite –
Returns:

new_instance() → int[source]¶

Description: New a NERICS Instance

The function must be invoked before mulitiple keyword scan filter

Parameters : Returns : NERICS_HANDLE: KeyScan Handle if success; otherwise return -1; Author : Kevin Zhang History :

1.create 2016-11-15

return:

delete_instance(handle: int) → int[source]¶

Parameters:	handle –
Returns:

import_doc(report_file: str, url_prefix: str = '', handle: int = 0) → str[source]¶

Func Name : NERICS_ImportDoc

Description: Read a Report file and save the result in file with XML format

Parameters : sReportFile: Report File

sURLPrefix: URL前缀路径 handle: NERICS handle, generated by NERICS_NewInstance

Returns : Return result file name: sXMLFile: XML file stored Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	report_file – url_prefix – handle –
Returns:

load_doc_result(result_xml_file: str, handle: int = 0) → int[source]¶

Func Name : NERICS_LoadDocResult

Description: Read a result XML file and save the result in file with XML format

Parameters : sReportFile: Report File

sURLPrefix: URL前缀路径 handle: NERICS handle, generated by NERICS_NewInstance

Returns : Return result file name: sXMLFile: XML file stored Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	result_xml_file – handle –
Returns:

check_report_f(report_file: str, url_prefix: str = '', organization: str = '', report_type: int = 0, format_opt: int = 1, handle: str = 0) → str[source]¶

Func Name : NERICS_CheckReportF

Description: Check a Report file and save the result in file with XML format

Parameters : sReportFile: Report File: 支持doc,docx,xml文件: sURLPrefix: URL前缀路径 nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance int nResultFormat：0: XML; 1:Jason

Returns : Return result file name: sXMLFile: XML file stored

Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	report_file – url_prefix – organization – report_type – format_opt – handle –
Returns:

check_report_m(report_text: str, url_prefix: str = '', organization: str = '', report_type: int = 0, format_opt: int = 1, handle: str = 0) → str[source]¶

Func Name : NERICS_CheckReportM

Description: Check a Report text memory and save the result in file with XML format

Parameters : sReportFile: Report File: 支持doc,docx,xml文件: sURLPrefix: URL前缀路径 nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance int nResultFormat：0: XML; 1:Jason

Returns : Return result file name: sXMLFile: XML file stored

Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	report_text – url_prefix – organization – report_type – format_opt – handle –
Returns:

extract_knowledge(report_text: str, report_type: int = 0) → str[source]¶

Func Name : NERICS_ExtractKnowledge

Description: Extract Knowledge from a text， given a configure string with XML format nType: Report Type, Default is RPT_UNSPECIFIC

Parameters : sReportFile: Report File: 支持doc,docx,xml文件: sURLPrefix: URL前缀路径 nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance int nResultFormat：0: XML; 1:Jason

Returns : Return result file name: sXMLFile: XML file stored

Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	report_text – report_type –
Returns:

get_result(result_type: int, handle: int = 0) → str[source]¶

Func Name : NERICS_GetResult

Description: 获取分析结果，默认为JSON格式

Parameters : result_type:: handle: NERICS handle, generated by NERICS_NewInstance

Returns : Return result file name: sXMLFile: XML file stored

Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	result_type – handle –
Returns:

add_audit_rule(audit_rule: str, report_type: int = 0) → int[source]¶

Func Name : NERICS_AddAuditRule

Description: Add Audit Rule

Parameters : sAuditRule: Audit rule,需要遵循KGB Audit语法规则: nType: Report Type, Default is RPT_UNSPECIFIC

Returns : int: 1: success, other: failed. Get error message via NERICS_GetLastErrorMsg()

Author : Kevin Zhang History :

1.create 2018-9-19

Parameters:	audit_rule – report_type –
Returns:

check_report_dir(report_dir: str, organization: str, report_type: int = 0, format_opt: int = 1, thread_count: int = 10) → str[source]¶

Func Name : NERICS_CheckReportDir

Description: Scan a dir and Check all doc files

Parameters : sReportDir: Report File Directory

nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance

Returns : Return result file name: sXMLFile: XML file stored Author : Kevin Zhang History :

1.create 2018-6-5

Parameters:	report_dir – organization – report_type – format_opt – thread_count –
Returns:

revise_report_f(revise_xml_file: str, handle: int = 0) → str[source]¶

Func Name : NERICS_ReviseReportF

Description: Revised a Report file: and revised information stored in file
Parameters : sReviseXMLFile: Revised information file with XML format: nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance

Returns : Return : new docx file name with path； return “” if failed!

Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	revise_xml_file – handle –
Returns:

show_html_error(revise_xml_file: str, handle: int = 0) → str[source]¶

Description: Revised a Report file

and revised information stored in file

Parameters : sReviseXMLFile: Revised information file with XML format: nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance

Returns : Return : new docx file name with path； return “” if failed!

Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	revise_xml_file – handle –
Returns:

import_template(template_file: str, report_type: int = 0, org: str = '', area: str = '', argument: str = '') → int[source]¶

Func Name : NERICS_ImportTemplate

Description: Import a document Template

Parameters : sTemplateFile: Template file using doc or docx format: nType: document type sOrg: organization sArgumemt: arguments
Returns : Return status: int: 1: success

Author : Kevin Zhang History :

1.create 2018-5-8

2.modified in 2018-11-20

Parameters:	template_file – report_type – org – area – argument –
Returns:

edit_template(template_id: int, template_file: str, report_type: int = 0, org: str = '', area: str = '', argument: str = '') → int[source]¶

Func Name : NERICS_EditTemplate

Description: Edit a document Template

Parameters : sTemplateFile: Template file using doc or docx format: nType: document type sOrg: organization sArgumemt: arguments
Returns : Return status: int: 1: success

Author : Kevin Zhang History :

1.create 2018-5-8

2.modified in 2018-11-20

Parameters:	template_id – template_file – report_type – argument – area – org –
Returns:

find_template(report_type: int = 0, org: str = '', area: str = '', argument: str = '') → int[source]¶

Func Name : NERICS_FindTemplate

Description: Find a document Template

Parameters :: nType: document type sOrg: organization sArgumemt: arguments
Returns : Return status: int: 1: success

Author : Kevin Zhang History :

1.create 2018-5-8

Parameters:	report_type – org – area – argument –
Returns:

delete_template(template_id: int) → int[source]¶

Func Name : NERICS_DeleteTemplate

Description: delete a document Template

Parameters : nTempID: template ID Returns : Return : int

Author : Kevin Zhang History :

1.create 2018-11-20

Parameters:	template_id –
Returns:

get_template(template_id: int) → str[source]¶

Func Name : NERICS_GetTemplate

Description: Get a document Template

Parameters : nTempID: template ID Returns : Return status: const char* :template data

Author : Kevin Zhang History :

1.create 2018-11-20

Parameters:	template_id –
Returns:

get_template_count(template_id: int) → str[source]¶

Func Name : NERICS_GetTemplateCount

Description: Get document Template count

Parameters : nTempID: template ID Returns : Return status: const char* :template data

Author : Kevin Zhang History :

1.create 2018-11-20

Parameters:	template_id –
Returns:

get_current_template_info(handle: int = 0) → str[source]¶

Func Name : NERICS_GetCurTemplateInfo

Description: Get current document Template information

Parameters : Returns : Return status: const char* :template information using Jason format

Author : Kevin Zhang History :

1.create 2018-12-5

Parameters:	handle –
Returns:

get_template_list(doc_type: int, organization: str) → ctypes.c_char_p[source]¶

Func Name : NERICS_GetTemplateList

Description: Get Template information

Parameters : docType: docType;: sOrgnization: organization name

Returns : Return status: const char* :template information using Jason format

Author : Kevin Zhang History :

1.create 2018-12-5

Parameters:	doc_type – organization –
Returns:

re_check_format(check_xml: str, template_id: int, format_opt: int = 1, handle: int = 0) → str[source]¶

Func Name : NERICS_ReCheckFormat

Description: ReCheck a format

Parameters : sReportFile: Report File: 支持doc,docx,xml文件: sURLPrefix: URL前缀路径 nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance int nResultFormat：0: XML; 1:Jason

Returns : Return result file name: sXMLFile: XML file stored

Author : Kevin Zhang History :

1.create 2018-11-27

Parameters:	check_xml – template_id – format_opt – handle –
Returns:

import_kgb_rules(rule_file: str, overwrite: bool = False, report_type: int = 0) → ctypes.c_int[source]¶

Func Name : NERICS_ImportKGBRules

Description: 针对报告类型nType导入相应的KGB规则集合

Parameters : sTemplateFile: Template file using doc or docx format Returns : Return status: int

1: success

Author : Kevin Zhang History :

1.create 2018-5-8

Parameters:	rule_file – overwrite – report_type –
Returns:

import_kgb_rules_from_mem(rule_text: str, overwrite: bool = False, report_type: int = 0) → int[source]¶

Func Name : NERICS_ImportKGBRulesFromMem

Description: 针对报告类型nType导入相应的KGB规则集合

Parameters : sTemplateFile: Template file using doc or docx format Returns : Return status: int

1: success

Author : Kevin Zhang History :

1.create 2018-5-8

:param rule_text :param overwrite: :param report_type: :return:

import_error_msg(error_list_file: str) → int[source]¶

Func Name : NERICS_ImportErrorMsg

Description: Import a error message table

Parameters : sErrorListFile: Template file using doc or docx format Returns : Return status: int

1: success

Author : Kevin Zhang History :

1.create 2018-5-8

Parameters:	error_list_file –
Returns:

import_sim_dict(sim_dict_file: str) → ctypes.c_int[source]¶

Func Name : NERICS_ImportSimDict

Description: Import simary dictionary

Parameters : sErrorListFile: Template file using doc or docx format Returns : Return status: int

1: success

Author : Kevin Zhang History :

1.create 2018-5-8

Parameters:	sim_dict_file –
Returns:

import_spell_error_dict(spell_error_dict: str) → int[source]¶

Func Name : NERICS_ImportSpellErrorDict

Description: Import Spelling Error dictionary

Parameters : sSpellErrorDict: Spelling Error dictionary Returns : Return status: int

1: success

Author : Kevin Zhang History :

1.create 2018-5-8

Parameters:	spell_error_dict –
Returns:

import_user_dict(user_dict: str)[source]¶

Func Name : NERICS_ImportUserDict

Description: Import Spelling Error dictionary

Parameters : sUserDict: User defined dictionary Returns : Return status: int

1: success

Author : Kevin Zhang History :

1.create 2019-12-3

Parameters:	user_dict –
Returns:

class nlpir.native.DeepClassifier(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

A dynamic link library native class for Classify using deep learning

FEATURE_COUNT = 800¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call DeepClassifier_Init

Init DeepClassifier

Parameters:	data_path – encode – license_code –
Returns:

exit_lib() → bool[source]¶

Call DeepClassifier_Init

Returns:

get_last_error_msg() → str[source]¶

Call DeepClassifier_GetLastErrorMsg

Returns:

new_instance(feature_count: int) → int[source]¶

Call DeepClassifier_NewInstance

New a DeepClassifier Instance. This function must be invoked before classify, and need be deleted when exit the process. Delete instance can use the function delete_instance()

Parameters:	feature_count – Feature count
Returns:	DeepClassifier Handle if success; otherwise return -1;

delete_instance(instance: int) → int[source]¶

Call DeepClassifier_DeleteInstance

Delete a DeepClassifier Instance with handle. The function must be invoked before release a specific classifier. The instance can be retrieve by new_instance()

Parameters:	instance – DeepClassifier Handle
Returns:

add_train(classname: str, text: str, handler: int = 0) → bool[source]¶

Call DeepClassifier_AddTrain

DeepClassifier add train dataset on given text in Memory

Parameters:	classname – class name text – text content handler – classifier handler
Returns:	add success or not

add_train_file(classname: str, filename: str, handler: int = 0) → int[source]¶

Call DeepClassifier_AddTrainFile

DeepClassifier add train dataset on given text in file

Parameters:	classname – class name filename – text file name handler – classifier handler
Returns:	success or fail

train(handler: int = 0) → int[source]¶

Call DeepClassifier_Train

DeepClassifier Training on given text in Memory. After training, the training result will stored. Then the classifier can load it with load_train_result() (offline or online).

Parameters:	handler – classifier handler
Returns:	success or not

load_train_result(handler: int = 0) → int[source]¶

Call DeepClassifier_LoadTrainResult

DeepClassifier Load already training data

Parameters:	handler – classifier handler
Returns:	success or not

export_features(filename: str, handler: int = 0) → int[source]¶

Call DeepClassifier_ExportFeatures

DeepClassifier Exports Features after training

Parameters:	filename – save path handler – classifier handler
Returns:	success or not

classify(text: str, handler: int = 0) → str[source]¶

Call DeepClassifier_Classify

DeepClassifier Classify on given text in Memory

Parameters:	text – text handler – classifier handler
Returns:	classify result , a class name

classify_ex(text: str, handler: nlpir.native.deep_classifier.LP_c_int = 0)[source]¶

Call DeepClassifier_ClassifyEx

DeepClassifier Classify on given text in Memory, return multiple class with weights, sorted by weights

Parameters:	text – text handler – classifier handler
Returns:	result with weight, For instance: `政治/1.20##经济/1.10,` `bookyzjs/7.00##bookxkfl/6.00##booktslx/5.00##bookny-xyfl/4.00##`

classify_file(filename: str, handler: int = 0)[source]¶

Call DeepClassifier_ClassifyFile

DeepClassifier Classify on given text in file

Parameters:	filename – file name of text handler – classifier handler
Returns:	result same as `classify()`

classify_file_ex(filename: str, handler: int = 0)[source]¶

Call DeepClassifier_ClassifyExFile

DeepClassifier Classify on given text in file

Parameters:	filename – file name of text handler – classifier handler
Returns:	result same as `classify_ex()`

class nlpir.native.DocExtractor(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

A dynamic link library native class for Document Extractor

DOC_EXTRACT_DELIMITER = '#'¶: 分隔符

DOC_EXTRACT_TYPE_MAX_LENGTH = 600¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call DE_Init

Parameters:	data_path (str) – encode (int) – license_code (str) –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call DE_Exit

Returns:	exit success or not

get_last_error_msg() → str[source]¶

Call DE_GetLastErrorMsg

Returns:	error message

pares_doc_e(text: str, user_def_pos: str, summary_needed: bool = True, func_required: int = 65535) → int[source]¶

Call DE_ParseDocE

生成单文档摘要

Parameters:	text – 文档内容 user_def_pos – 用户自定义的词性标记, 最多三种（人名、地名、机构名、媒体等内置，无需设置, 不同词类之间采用#分割, 如 `gms#gjtgj#g` summary_needed – 是否需要计算摘要 func_required –
Returns:	用于获取内容的handle, 获取内容完毕后应使用 `release_handle()` 释放对应资源

release_handle(handle: int) → None[source]¶

Call DE_ReleaseHandle

释放 parse_doc_e() 结果所占据的空间

Parameters:	handle – `parse_doc_e()` 执行后返回的HANDLE
Returns:

get_result(handle: int, doc_extract_type: int) → str[source]¶

Call DE_GetResult

从运行完的 parse_doc_e() 结果中，获取指定抽取的结果内容

Parameters:	handle – `parse_doc_e()` 执行后返回的HANDLE doc_extract_type – 获取的抽取类型，从DOC_EXTRACT_TYPE_PERSON开始的结果
Returns:

get_sentiment_score(handle: int) → int[source]¶

Call DE_GetSentimentScore

从运行完的 parse_doc_e() 结果中，获取指文章的情感得分

Parameters:	handle – `parse_doc_e()` 执行后返回的HANDLE
Returns:	情感正负得分

compute_sentiment_doc(text: str) → int[source]¶

Call DE_ComputeSentimentDoc

生成单文档情感分析结果

Parameters:	text – 文档内容
Returns:

import_sentiment_dict(filename: str) → int[source]¶

Call DE_ImportSentimentDict

导入用户自定义的情感词表，每行一个词，空格后加上正负权重，如: 语焉不详 -2

若导入的情感词属于新词, 需先在用户词典中导入, 否则情感识别自动跳跃

Parameters:	filename –
Returns:

import_user_dict(filename: str, overwrite: bool = False) → int[source]¶

Call DE_ImportUserDict

导入用户词典, see nlpir.native.ictclas.ICTCLAS.import_user_dict()

Parameters:	filename – overwrite –
Returns:

add_user_word(word: str) → int[source]¶

Call DE_AddUserWord

Add a word to the user dictionary, see nlpir.native.ictclas.ICTCLAS.add_user_word()

Parameters:	word –
Returns:

clean_user_word() → int[source]¶

Call DE_CleanUserWord

Clean all temporary added user words, see nlpir.native.ictclas.ICTCLAS.clean_user_word()

Returns:

save_the_usr_dic() → int[source]¶

Call DE_SaveTheUsrDic

Save in-memory dict to user dict, see nlpir.native.ictclas.ICTCLAS.save_the_usr_dic() :return:

del_usr_word(word: str) → int[source]¶

Call DE_DelUsrWord

Delete a word from the user dictionary, see nlpir.native.ictclas.ICTCLAS.del_usr_word()

Parameters:	word –
Returns:

import_key_blacklist(filename: str, pos_blacklist: str) → int[source]¶

Call DE_ImportKeyBlackList

Import keyword black list, see nlpir.native.key_extract.KeyExtract.import_key_blacklist()

Parameters:	filename – pos_blacklist –
Returns:

class nlpir.native.SentimentNew(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call ST_Init

Parameters:	data_path – encode – license_code –
Returns:

exit_lib() → bool[source]¶

Call ST_Exit

Returns:

get_last_error_msg() → str[source]¶

Call ST_GetLastErrorMsg

Returns:

get_one_object_result(title: str, content: str, analysis_object: str) → str[source]¶

Call ST_GetOneObjectResult

Parameters:	title – content – analysis_object –
Returns:

get_multi_object_result(title: str, content: str, object_rule_file: str) → str[source]¶

Call ST_GetMultiObjectResult

Parameters:	title – content – object_rule_file – see Appendix II: Multiple Object configure sample
Returns:

get_sentence_point(sentence: str) → str[source]¶

Call ST_GetSentencePoint

Get multiple object sentimental result

Parameters:	sentence –
Returns:	double,Sentimental point

get_sentiment_point(sentence: str) → float[source]¶

Call ST_GetSentimentPoint

Get multiple object sentimental result

Parameters:	sentence –
Returns:	double,Sentimental point

import_user_dict(filename: str, over_write: bool = False) → int[source]¶

Call ST_ImportUserDict

Import User-defined dictionary, same as nlpir.native.ictclas.ICTCLAS.import_user_dict()

Parameters:	filename – over_write –
Returns:

process_dir(path: str) → str[source]¶

Call ST_ProcesDir

批量处理指定的目录下的文本文件. 分析结果, 输出到指定的Excel文件中

Parameters:	path –
Returns:	path目录下, 自动生成 `SentimentRankResult.xls`,返回该文件的全路径名称

class nlpir.native.SentimentAnalysis(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

EMOTION_HAPPY = 0¶

EMOTION_GOOD = 1¶

EMOTION_ANGER = 2¶

EMOTION_SORROW = 3¶

EMOTION_FEAR = 4¶

EMOTION_EVIL = 5¶

EMOTION_SURPRISE = 6¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call LJST_Init

Parameters:	data_path – encode – license_code –
Returns:

exit_lib() → bool[source]¶

Call LJST_Exits

Returns:

get_last_error_msg() → str[source]¶

Returns:

get_paragraph_sent(paragraph: str) → Tuple[bool, str][source]¶

Call LJST_GetParagraphSent

Get sentiment analyze result

Parameters:	paragraph –
Returns:

get_file_sent(filename: str) → Tuple[bool, str][source]¶

Call LJST_GetFileSent

Get sentiment analyze result

Parameters:	filename –
Returns:

import_user_dict(filename: str, over_write: bool = False)[source]¶

Call LJST_ImportUserDict

Import User-defined dictionary, same as nlpir.native.ictclas.ICTCLAS.import_user_dict()

Parameters:	filename – over_write –
Returns:

get_paragraph_sent_e(paragraph: str) → str[source]¶

Call LJST_GetParagraphSentE

Get sentiment analyze result

Parameters:	paragraph –
Returns:

get_file_sent_e(filename: str) → str[source]¶

Call LJST_GetFileSentE

Get sentiment analyze result

Parameters:	filename –
Returns:

class nlpir.native.Summary(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

load_mode = 1¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call DS_Init

Parameters:	data_path – encode – license_code –
Returns:

exit_lib() → bool[source]¶

Call DS_Exit

Returns:

get_last_error_msg() → str[source]¶

Call DS_GetLastErrMsg

Returns:

single_doc(text: str, sum_rate: float = 0.0, sum_len: int = 500, html_tag_remove: int = 0, sentence_count: int = 0)[source]¶

Call DS_SingleDoc

生成单文档摘要, make summarization

Parameters:

text (str) – 文档内容 text content
sum_rate (float) – 文档摘要占原文百分比(为0.00则不限制） the percentage of summarization length comparing to original text (0.00 represent no limit)
sum_len (int) – 用户限定的摘要长度(为0则不限制）The max len of summarization(0 will no limit)
html_tag_remove (bool) – 是否需要对原文进行Html标签的去除 remove the html tag or not
sentence_count (int) – 用户限定的句子数量（为0则不限制）

Returns:

摘要字符串；出错返回空串 the summarization content, get null string if occurs error.

single_doc_e(text: str, sum_rate: float = 0.0, sum_len: int = 500, html_tag_remove: int = 0, sentence_count: int = 0)[source]¶

Call DS_SingleDocE

生成单文档摘要该函数支持多线程，是多线程安全的, make summarization with threading safe

Parameters:

text (str) – 文档内容 text content
sum_rate (float) – 文档摘要占原文百分比(为0.00则不限制） the percentage of summarization length comparing to original text (0.00 represent no limit)
sum_len (int) – 用户限定的摘要长度(为0则不限制）The max len of summarization(0 will no limit)
html_tag_remove (bool) – 是否需要对原文进行Html标签的去除 remove the html tag or not
sentence_count (int) – 用户限定的句子数量（为0则不限制）

Returns:

摘要字符串；出错返回空串 the summarization content, get null string if occurs error.

file_process(text_filename: str, sum_rate: float = 0.0, sum_len: int = 500, html_tag_remove: int = 0, sentence_count: int = 0)[source]¶

Call DS_FileProcess

生成单文档摘要该函数支持多线程,是多线程安全的, make summarization from file with threading safe

Parameters:

text_filename (str) – 文档文件路径 text file path
sum_rate (float) – 文档摘要占原文百分比(为0.00则不限制） the percentage of summarization length comparing to original text (0.00 represent no limit)
sum_len (int) – 用户限定的摘要长度(为0则不限制）The max len of summarization(0 will no limit)
html_tag_remove (bool) – 是否需要对原文进行Html标签的去除 remove the html tag or not
sentence_count (int) – 用户限定的句子数量（为0则不限制）

Returns:

摘要字符串；出错返回空串 the summarization content, get null string if occurs error.

class nlpir.native.KeyExtract(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

A dynamic link library native class for Key Words Extract

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call KeyExtract_Init

Parameters:	data_path (str) – encode (int) – license_code (str) –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call KeyExtract_Exit

Returns:	exit success or not

get_keywords(line: str, max_key_limit: int = 50, format_opt: int = 0) → str[source]¶

Call KeyExtract_GetKeyWords

Extract keyword from text, 从文本中获取关键词

Parameters:	line – the input paragraph max_key_limit – maximum of key words, up to 50 format_opt – output format option, there three options: `nlpir.native.nlpir_base.OUTPUT_FORMAT_SHARP` get string split by sharp `nlpir.native.nlpir_base.OUTPUT_FORMAT_JSON` get json format `nlpir.native.nlpir_base.OUTPUT_FORMAT_EXCEL` get csv format
Returns:	the keyword with weight

Split with #:

科学发展观/n/23.80/12#宏观经济/n/12.20/12#

JSON形式:

[
    {
        'freq': 2,
        'pos': 'n_new',
        'weight': 7.771335980376418,
        'word': '国家权力'
    },{
        'freq': 7,
        'pos': 'n',
        'weight': 7.438759706600493,
        'word': '权力'
    },{
        'freq': 1,
        'pos': 'nrf',
        'weight': 5.280000338096665,
        'word': '孟德斯鸠'
    },{ ...
    }, ...
]

get_file_keywords(filename: str, max_key_limit: int = 50, format_opt: int = 0) → str[source]¶

Call KeyExtract_GetKeyWords

Extract keyword from file, 从文本文件中获取关键词

Parameters:	filename – the input text file max_key_limit – maximum of key words, up to 50 format_opt – same as `get_keywords()`
Returns:	the keyword with weight

Split with #

科学发展观/n/23.80/12#宏观经济/n/12.20/12#

JSON形式:

[
    {
        'freq': 2,
        'pos': 'n_new',
        'weight': 7.771335980376418,
        'word': '国家权力'
    },{
        'freq': 7,
        'pos': 'n',
        'weight': 7.438759706600493,
        'word': '权力'
    },{
        'freq': 1,
        'pos': 'nrf',
        'weight': 5.280000338096665,
        'word': '孟德斯鸠'
    },{ ...
    }, ...
]

import_user_dict(filename: str, overwrite: bool = False)[source]¶

Call KeyExtract_ImportUserDict

Import a user dict to the system, the format of the dict file:

word1 pos_tag
word2 pos_tag

If you import a user dict to the system, the user dict will save to the system (in Data directory). You cannot delete the word in the user dict from the system use clean_user_word() or del_usr_word().

Parameters:	filename (str) – the path of user dict file overwrite (bool) – overwrite the current user dict or not
Returns:	import success or not 1->True 2->False

add_user_word(word: str) → int[source]¶

Call KeyExtract_AddUserWord

Add a word to the user dictionary ,example:

单词 词性

or:

单词 (default n)

The added word only add in memory and will not affect the user dict, you can use clean_user_word() or del_usr_word() to delete the word or all the words in memory. If you want to save to the user dict ,use save_the_usr_dic() to save to the Data directory.

Parameters:	word (str) –
Returns:	1,true ; 0,false

clean_user_word() → int[source]¶

Call KeyExtract_CleanUserWord

Clean all temporary added user words, more info see add_user_word()

Returns:	1,true ; 0,false

clean_current_user_word() → int[source]¶

Call KeyExtract_CleanCurrentUserWord Clean all Current temporary added user words and restore previous stored data

** Now Only for win and linux x64 **

Returns:	1,true ; 0,false

save_the_usr_dic() → int[source]¶

Call KeyExtract_SaveTheUsrDic

Save in-memory dict to user dict, more info see add_user_word()

Returns:	1,true; 2,false

del_usr_word(word: str) → int[source]¶

Call KeyExtract_DelUsrWord

Delete a word from the user dictionary, more info see add_user_word()

Parameters:	word (str) – the word to be delete
Returns:	-1, the word not exist in the user dictionary; else, the handle of the word deleted

import_key_blacklist(filename: str, pos_blacklist: Optional[str] = None) → int[source]¶

Call KeyExtract_ImportKeyBlackList

Import keyword black list

This function will save words to KeyBlackList.pdat , if you want to remove the words form the system need to backup it before use this function. Or use the function nlpir.key_extract.import_blacklist() , That function will backup that file automatically and you can use nlpir.key_extract.clean_blacklist() to clean current blacklist and restore the origin file.

This list of word will not affect the key word extract and segmentation

Parameters:	filename – A word list that the words want to import to the blacklist (stop word list), 一个停用词词表,里面为想进行屏蔽的词,也可以包括别的词,是否不进行抽取是按照词表中的词性来确定的. pos_blacklist – A list of pos that want to block in the system, 想要屏蔽的词的词性
Returns:	number of words that import to the systems

batch_start() → int[source]¶

Call KeyExtract_Batch_Start

启动关键词识别

Returns:	rue:success, false:fail

batch_add_file(filename) → int[source]¶

Call KeyExtract_Batch_AddFile

往关键词识别系统中添加待识别关键词的文本文件, 需要在运行 batch_start() 之后，才有效

Parameters:	filename – 文件名
Returns:	true:success, false:fail

batch_addmen(txt: str) → bool[source]¶

Call KeyExtract_Batch_AddMem

往关键词识别系统中添加一段待识别关键词的内存,需要在运行 batch_start() 之后，才有效

Parameters:	txt – 文件名
Returns:	true:success, false:fail

batch_complete() → int[source]¶

Call KeyExtract_Batch_Complete

关键词识别添加内容结束,需要在运行 batch_start() 之后，才有效

Returns:	true:success, false:fail

batch_getresult(weight_out: bool) → str[source]¶

Call KeyExtract_Batch_GetResult

获取关键词识别的结果,需要在运行 batch_complete() 之后，才有效

Parameters:	weight_out – 是否需要输出每个关键词的权重参数
Returns:	输出格式为【关键词1】【权重1】【关键词2】【权重2】 …

get_last_error_msg() → str[source]¶

Call KeyExtract_GetLastErrorMsg

Returns:	error message

class nlpir.native.KeyScanner(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

A dynamic link library native class for Keyword Scan

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call KS_Init

Parameters:	data_path (str) – encode (int) – license_code (str) –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call KS_Exit

Returns:	exit success or not

get_last_error_msg() → str[source]¶

Call KS_GetLastErrorMsg

Returns:	error message

new_instance(filter_type_index: int = 0) → int[source]¶

Call KS_NewInstance

Get a instance from system for executing other functions. The function must be invoked before multiple keyword scan filter. This function will alloc memory , it need to be free memory by using delete_instance() after finish all executions from this handle.

Parameters:	filter_type_index – which No of filter want to be used in this instance. The filter file will save into Data/KeyScanner/filter{no}*
Returns:	a handle from system if success; otherwise return -1;

delete_instance(handle: int) → int[source]¶

Call KS_DeleteInstance

Delete handle created by :func`new_instance`. Once delete handle from system, this handle cannot be used in any situation or will invoke critical errors.

Parameters:	handle – the handle want to be deleted
Returns:	success or not

import_user_dict(filename: str, over_write: bool = False, pinyin_abbrev_needed: bool = False, handle=0) → int[source]¶

Call ImportUserDict

Import User-defined dictionary 导入用户词典, 此操作为全局操作会影响其他 instance 的过滤

文本文件每行的格式为: 词条词类权重 (注意，最多定义255个类别), 例:

AV电影 色情 2
六合彩 涉赌 8 1

复杂过滤条件: 支持与或非处理 ;表示或关系，+表示与关系，-表示否格式如下:

{key11;key12;key13;...;key1N}+{key21;key22;key23;...;key2N}+...+{keyM1;keyM2;keyM3;...;keyMN}-{keyN}

示例:

{中国;中华;中华人民共和国;中国共产党;中共}+{伟大;光荣;正确}-{中华民国;国民党}  政治类 5

表示的是文本内容中包含 中国;中华;中华人民共和国;中国共产党;中共 中的一种，同时出现 伟大;光荣;正确 中的一个，但不能出现 中华民国;国民党 的任何一个

Parameters:	filename – path of user dictionary pinyin_abbrev_needed – over_write – true将覆盖系统已经有的词表；否则将采用追加的方式追加不良词表 handle – handle of KeyScanner
Returns:	success or not

delete_user_dic(text: str, handle: int) → int[source]¶

Call DeleteUserDict

Delete User-defined dictionary 删除用户词典, 此操作为全局操作, 会删除词典文件并影响所有 instance

文本文件每行的格式为: 词条 , 例如:

AV电影
习近平

Parameters:	text – Text of user dictionary handle – handle of KeyScanner
Returns:	The number of lexical entry deleted successfully 成功删除的词典条数

delete_user_dic_from_file(filename: str, handle: int) → int[source]¶

Call DeleteUserDict

Delete User-defined dictionary 删除用户词典, 此操作为全局操作, 会删除词典文件并影响所有 instance

文本文件每行的格式为: 词条 , 例如:

AV电影
习近平

Parameters:	filename – Text filename for user dictionary handle – handle of KeyScanner
Returns:	The number of lexical entry deleted successfully 成功删除的词典条数

scan(content: str, handle: int = 0) → str[source]¶

Call KS_Scan

扫描输入的文本内容

Parameters:	content – 文本内容 handle – handle of KeyScanner
Returns:	涉及不良的所有类别与权重，按照权重排序。如: `色情/10#暴力/1#` , `政治反动/2#FLG/1#涉领导人/1#` , `""` : 表示无扫描命中结果

scan_detail(content: str, scan_mode: int = 0, handle: int = 0) → str[source]¶

Call KS_ScanDetail

扫描输入的文本内容,获得详细结果

Parameters:	scan_mode – 扫描模式 content – 文本内容 handle – handle of KeyScanner
Returns:	返回包含了扫描结果的内容，扫描结果明细:

{
    "Details": ["chou傻逼xi禁评"],
    "Rules": ["傻逼","xi禁评"],
    "filename": "",
    "illegal" :{
        "classes":[
            {
                "freq":1,
                "word":"粗言秽语"
            },{
                "freq":1,
                "word":"污言秽语"
            },{
                "freq":1,
                "word":"新华社禁用"
            },{
                "freq":1,"word":"一号首长"
            }
        ],
        "hit_count":4,
        "keys":["傻逼","xi禁评"],
        "scan_val":13.333333333333332
    },
    "legal": {
        "hit_count":0,
        "scan_val":0.0
    },
    "line_id":0,
    "org_file":"",
    "score":13.333333333333332
}

scan_file(filename: str, handle: int = 0) → str[source]¶

Call KS_ScanFile

扫描输入的文本文件内容

Parameters:	filename – 文本文件名 handle – handle of KeyScanner
Returns:	same as `scan()`

scan_file_detail(filename: str, handle: int = 0) → str[source]¶

Call KS_ScanFileDetail

扫描输入的文本文件内容

Parameters:	filename – 文本文件名 handle – handle of KeyScanner
Returns:	same as `scan_detail()`

scan_line(filename: str, result_filename: str, handle: int = 0, encrypt: int = 0, scan_mode: int = 0) → int[source]¶

Call KS_ScanLine

按行扫描输入的文本文件内容

Parameters:	filename – 输入的文本文件名 result_filename – 输出的结果文件名 handle – handle of KeyScanner encrypt – 0 不加密；1，加密 scan_mode –
Returns:	same as `scan_detail()`

scan_stat(result_file, handle: int = 0) → int[source]¶

Call KS_ScanStat

输出扫描结果的命中统计报告，利于进一步的分析核查

Parameters:	result_file – 输出结果的文件文件 handle – handle of KeyScanner
Returns:	成功扫描到问题的文件数

scan_dir(input_dir_path: str, result_path: str, filter: str, thread_count: int = 10, encrypt: bool = False, scan_mode: int = 0) → int[source]¶

Call KS_ScanDir

多线程扫描按行扫描输入的文本夹文件内容

Parameters:	input_dir_path – 输入的文件夹路径 result_path – 输出结果的文件夹路径 filter – 输入的文件后缀名 thread_count – 线程数，默认10个 encrypt – 0 不加密；1，加密 scan_mode –
Returns:	成功扫描到问题的文件数

merge_result(path: str) → None[source]¶

Merge多线程的扫描结果

Parameters:	path –
Returns:

scan_add_stat(result_file: str, handle: int) → int[source]¶

将handle线程扫描结果归并到0线程

Parameters:	result_file – handle –
Returns:

stat_result_filter(input_filename: str, result_filename: str, threshold: float = 5.0) → int[source]¶

Call KS_StatResultFilter

对扫描的统计结果进行过滤分析

Parameters:	input_filename – 输入的结果文件名 result_filename – 输出结果的文件名 threshold – 不良得分的阈值
Returns:	成功扫描到问题的文件数

scan_result_filter(input_filename: str, result_filename: str, threshold: float = 9.0) → int[source]¶

Call KS_ScanResultFilter

对扫描的详细结果文件进行过滤分析

Parameters:	input_filename – 输入的结果文件名 result_filename – 输出结果的文件名 threshold – 不良得分的阈值
Returns:	成功扫描到问题的文件数

decrypt(input_dir_path: str, result_path: str) → int[source]¶

Call KS_Decrypt

多线程转换扫描结果

Parameters:	input_dir_path – 输入的文件夹路径 result_path – 输出结果的文件夹路径
Returns:

export_dict(filename: str, handle: int = 0) → int[source]¶

Call KS_ExportDict

ExportDict dictionary 导出已经定义的不良词词典, 为保护知识产权，该功能仅局限于管理员内部调度使用

文本文件的格式为: 词条词类权重 (注意，最多定义255个类别) 例如:

AV电影 色情 2
六合彩 涉赌 8 1

Parameters:	filename – Text filename for user dictionary handle – handle of KeyScanner
Returns:	The number of lexical entry imported successfully 成功导入的词典条数

class nlpir.native.TextSimilarity(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call TS_Init

Parameters:	data_path – encode – license_code –
Returns:

exit_lib() → bool[source]¶

Call DS_Exit

Returns:

get_last_error_msg() → str[source]¶

Call TS_GetLastErrorMsg

Returns:

compute_sim(text_1: str, text_2: str, model: int = 2) → float[source]¶

Call TS_ComputeSim

Parameters:	text_1 – text_2 – model –
Returns:

compute_sim_file(filename_1: str, filename_2: str, model: int = 2) → float[source]¶

Call TS_ComputeSimFile

Parameters:	filename_1 – filename_2 – model –
Returns:

class nlpir.native.NewWordFinder(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call NWF_Init

Parameters:	data_path (str) – encode (int) – license_code (str) –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call NWF_Exit

Returns:	exit success or not

get_new_words(line: str, max_key_limit: int = 50, format_opt: int = 0) → str[source]¶

Call NWF_GetNewWords

Extract New words from line

Parameters:	line (str) – the input paragraph

The input size cannot be very big(less than 60MB). Process large memory, recommend use NWF_NWI series functions

Parameters:	max_key_limit (str) – maximum of key words, up to 50 format_opt (int) – output format option, there three options: `nlpir.native.nlpir_base.OUTPUT_FORMAT_SHARP` get string split by sharp `nlpir.native.nlpir_base.OUTPUT_FORMAT_JSON` get json format `nlpir.native.nlpir_base.OUTPUT_FORMAT_EXCEL` get csv format
Returns:	new words list

Sharp format
"科学发展观/23.80/1#屌丝/12.20/2" with weight
Json格式如下：
[
   {
      "freq" : 152,
      "pos" : "n_new",
      "weight" : 77.884208081632579,
      "word" : "公允价值"
   },
   {
      "freq" : 71,
      "pos" : "n_new",
      "weight" : 75.102183562405372,
      "word" : "长期股权投资"
   }
]

get_file_new_words(file_name: str, max_key_limit: int = 50, format_opt: int = 0) → str[source]¶

Call NWF_GetFileNewWords

Extract new words from a text file

Parameters:	file_name (str) – the path of text file max_key_limit (int) – max key want to get format_opt (int) – same as `get_new_words()`
Returns:	same as `get_new_words()`

batch_start() → bool[source]¶

Call NWF_Batch_Start

启动新词识别,for very large size of data

Returns:	true:success, false:fail

batch_addfile(filename: str) → int[source]¶

Call NWF_Batch_AddFile

往新词识别系统中添加待识别新词的文本文件,需要在运行NWF_Batch_Start()之后，才有效

Parameters:	filename (str) – the path of file
Returns:	1 success 0 fail

batch_addmen(text: str) → int[source]¶

Call NWF_Batch_AddMem

往新词识别系统中添加一段待识别新词的内存,需要在运行NWF_Batch_Start()之后，才有效

Parameters:	text (str) – text string
Returns:	1 success 0 fail

batch_complete() → int[source]¶

Call NWF_Batch_Complete

新词识别添加内容结束,需要在运行NWF_Batch_Start()之后，才有效

Returns:	1 success 0 fail

batch_getresult(format_json: bool = False) → str[source]¶

Call NWF_Batch_GetResult

获取新词识别的结果, 需要在运行NWF_Batch_Complete()之后，才有效

Parameters:	format_json (bool) – get json format or not
Returns:	输出格式为

新词1】 【权重1】 【新词2】 【权重2】 ...
Json格式如下：
[
   {
      "freq" : 152,
      "pos" : "n_new",
      "weight" : 77.884208081632579,
      "word" : "公允价值"
   },
   {
      "freq" : 71,
      "pos" : "n_new",
      "weight" : 75.102183562405372,
      "word" : "长期股权投资"
   }
]

result2user_dict() → int[source]¶

Call NWF_Result2UserDict

将新词识别结果导入到用户词典中,需要在运行NLPIR_NWI_Complete()之后，才有效. 如果需要将新词结果永久保存，建议在执行NLPIR_SaveTheUsrDic

Returns:	bool, true:success, false:fail

get_last_error_msg() → str[source]¶: 对应每个组件获取异常信息的函数

Submodules¶

nlpir.native.ictclas module¶

class nlpir.native.ictclas.ResultT[source]¶

Bases: _ctypes.Structure

The NLPIR result_t structure. copy from pynlpir

iPOS¶: Structure/Union member

length¶: Structure/Union member

sPOS¶: Structure/Union member

start¶: Structure/Union member

weight¶: Structure/Union member

word_ID¶: Structure/Union member

word_type¶: Structure/Union member

class nlpir.native.ictclas.ICTCLAS(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

A dynamic link library native class for Chinese Segmentation

POS_MAP_NUMBER = 4¶

ICT_POS_MAP_FIRST = 1¶

ICT_POS_MAP_SECOND = 0¶

PKU_POS_MAP_SECOND = 2¶

PKU_POS_MAP_FIRST = 3¶

POS_SIZE = 40¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call NLPIR_Init

Parameters:	data_path (str) – encode (int) – license_code (str) –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call NLPIR_Exit

Returns:	exit success or not

paragraph_process(paragraph: str, pos_tagged: int = 1) → str[source]¶

Call NLPIR_ParagraphProcessing

Chinese word segment, segment paragraph to a string

Parameters:	paragraph (str) – the string want to be segmented pos_tagged (int) – show the pos tag or not 1-> True, 0-> False
Returns:	segmented string

paragraph_process_a(paragraph: str, user_dict: bool = True) → Tuple[nlpir.native.ictclas.ResultT, int][source]¶

Call ParagraphProcessingA

Segment paragraph to an Array of ResultT, get more detail info

Parameters:	paragraph (str) – the string want to be segmented user_dict (bool) – use user dictionary or not
Returns:	a result of segment, an array of ResultT and the length of the ResultT

get_paragraph_process_a_word_count(paragraph: str) → int[source]¶

paragraph_process_aw(count: int, result: nlpir.native.ictclas.ResultT) → None[source]¶

file_process(source_filename: str, result_filename: str, pos_tagged: int = 1) → float[source]¶

Call NLPIR_FileProcess

Segment a text file and save to a file.

Parameters:	source_filename (str) – the path of a text file that want to be segmented result_filename (str) – the path to save the result of segmentation pos_tagged (int) – show the pos tag or not 1->True, 0->False

import_user_dict(filename: str, overwrite: bool = False) → int[source]¶

Call NLPIR_ImportUserDict

Import a user dict to the system, the format of the dict file:

word1 pos_tag
word2 pos_tag

If you import a user dict to the system, the user dict will save to the system (in Data directory). You cannot delete the word in the user dict from the system use clean_user_word() or del_usr_word().

TODO add more comment for clean the user dict and add the function to the high-level method

Parameters:	filename (str) – the path of user dict file overwrite (bool) – overwrite the current user dict or not
Returns:	import success or not 1->True 2->False

add_user_word(word: str) → int[source]¶

Call NLPIR_AddUserWord

Add a word to the user dictionary ,example:

单词 词性

or:

单词 (default n)

The added word only add in memory and will not affect the user dict, you can use clean_user_word() or del_usr_word() to delete the word or all the words in memory. If you want to save to the user dict ,use save_the_usr_dic() to save to the Data directory.

Parameters:	word (str) –
Returns:	1,true ; 0,false

clean_user_word() → int[source]¶

Call NLPIR_CleanUserWord

Clean all temporary added user words, more info see add_user_word() TODO figure out the return value :return: 1,true ; 0,false

clean_current_user_word() → int[source]¶

Call NLPIR_CleanCurrentUserWord Clean all Current temporary added user words and restore previous stored data

** Now Only for win and linux x64 **

Returns:	1,true; 2,false

save_the_usr_dic() → int[source]¶

Call NLPIR_SaveTheUsrDic

Save in-memory dict to user dict, more info see add_user_word()

Returns:	1,true; 2,false

del_usr_word(word: str) → int[source]¶

Call NLPIR_DelUsrWord

Delete a word from the user dictionary, more info see add_user_word()

Parameters:	word (str) – the word to delete
Returns:	-1, the word not exist in the user dictionary; else, the handle of the word deleted

get_uni_prob(word) → float[source]¶

Call NLPIR_GetUniProb

Get Unigram Probability

Parameters:	word (str) – input word
Returns:	The unitary probability of a word.

is_word(word: str) → int[source]¶

Call NLPIR_IsWord

Judge whether the word is included in the core dictionary

Parameters:	word (str) – input word
Returns:	1: exists; 0: no exists

is_user_word(word: str, is_ascii: bool = False) → int[source]¶

Call NLPIR_IsUserWord

Judge whether the word is included in the user-defined dictionary

Parameters:	word (str) – input word is_ascii (bool) – is ascii encode or not
Returns:	1: exists; 0: no exists

get_word_pos(word: str) → str[source]¶

Call NLPIR_GetWordPOS

Get the word Part-Of-Speech information

Parameters:	word (str) – input word
Returns:	pos tagging

set_pos_map(pos_map: int) → int[source]¶

Call NLPIR_SetPOSmap

Select which pos map will use:

ICT_POS_MAP_FIRST 计算所一级标注集
ICT_POS_MAP_SECOND 计算所二级标注集
PKU_POS_MAP_SECOND 北大二级标注集
PKU_POS_MAP_FIRST 北大一级标注集

Default is ICT_POS_MAP_SECOND

Parameters:	pos_map (int) –
Returns:	0, failed; else, success

finer_segment(line: str) → str[source]¶

Call NLPIR_FinerSegment

当前的切分结果过大时,如“中华人民共和国”, 需要执行该函数,将切分结果细分为“中华人民共和国”

细分粒度最大为三个汉字,如果不能细分，则返回为空字符串

Parameters:	line (str) – string need to be segmented
Returns:	segmented string, return null string if line cannot be segmented

get_eng_word_origin(word: str) → str[source]¶

Call NLPIR_GetEngWordOrign

获取各类英文单词的原型，考虑了过去分词、单复数等情况:

driven->drive   drives->drive   drove-->drive

Parameters:	word (str) – word to be stemmed
Returns:	the stemmed word

word_freq_stat(text: str, stop_word_remove: bool = True) → str[source]¶

Call NLPIR_WordFreqStat

获取输入文本的词，词性，频统计结果，按照词频大小排序

Parameters:	text (str) – 输入的文本内容 stop_word_remove (bool) – true-去除停用词 false-不去除停用词
Returns:	返回的是词频统计结果形式如下

张华平/nr/10#博士/n/9#分词/n/8

file_word_freq_stat(filename: str, stop_word_remove: bool = True) → str[source]¶

Call NLPIR_FileWordFreqStat

Same as word_freq_stat()

Parameters:	filename (str) – path of text file stop_word_remove (bool) – remove stop words or not
Returns:	same as `word_freq_stat()`

get_last_error_msg() → str[source]¶

Call NLPIR_GetLastErrorMsg

Returns:	error message

tokenizer_for_ir(text: str, fine_segment: bool = False) → str[source]¶

Call NLPIR_Tokenizer4IR

搜索引擎模式，在精确模式的基础上，对长词再次切分，提高召回率，适合用于搜索引擎分词

Parameters:	text (str) – The source paragraph fine_segment (bool) – Need finer segment or not
Returns:

输入：国务院办公厅转发商务部的结果如下：

[
    {
    "begin" : 0,
    "end" : 6,
    "pos" : "nt",
    "text" : "国务院办公厅"
    },
    {
    "begin" : 0,
    "end" : 3,
    "pos" : "",
    "text" : "国务院"
    },
    {
    "begin" : 3,
    "end" : 6,
    "pos" : "",
    "text" : "办公厅"
    },
    {
    "begin" : 6,
    "end" : 8,
    "pos" : "v",
    "text" : "转发"
    },
    {
    "begin" : 8,
    "end" : 11,
    "pos" : "n",
    "text" : "商务部"
    }
]

nlpir.native.new_word_finder module¶

class nlpir.native.new_word_finder.NewWordFinder(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call NWF_Init

Parameters:	data_path (str) – encode (int) – license_code (str) –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call NWF_Exit

Returns:	exit success or not

get_new_words(line: str, max_key_limit: int = 50, format_opt: int = 0) → str[source]¶

Call NWF_GetNewWords

Extract New words from line

Parameters:	line (str) – the input paragraph

The input size cannot be very big(less than 60MB). Process large memory, recommend use NWF_NWI series functions

Parameters:	max_key_limit (str) – maximum of key words, up to 50 format_opt (int) – output format option, there three options: `nlpir.native.nlpir_base.OUTPUT_FORMAT_SHARP` get string split by sharp `nlpir.native.nlpir_base.OUTPUT_FORMAT_JSON` get json format `nlpir.native.nlpir_base.OUTPUT_FORMAT_EXCEL` get csv format
Returns:	new words list

Sharp format
"科学发展观/23.80/1#屌丝/12.20/2" with weight
Json格式如下：
[
   {
      "freq" : 152,
      "pos" : "n_new",
      "weight" : 77.884208081632579,
      "word" : "公允价值"
   },
   {
      "freq" : 71,
      "pos" : "n_new",
      "weight" : 75.102183562405372,
      "word" : "长期股权投资"
   }
]

get_file_new_words(file_name: str, max_key_limit: int = 50, format_opt: int = 0) → str[source]¶

Call NWF_GetFileNewWords

Extract new words from a text file

Parameters:	file_name (str) – the path of text file max_key_limit (int) – max key want to get format_opt (int) – same as `get_new_words()`
Returns:	same as `get_new_words()`

batch_start() → bool[source]¶

Call NWF_Batch_Start

启动新词识别,for very large size of data

Returns:	true:success, false:fail

batch_addfile(filename: str) → int[source]¶

Call NWF_Batch_AddFile

往新词识别系统中添加待识别新词的文本文件,需要在运行NWF_Batch_Start()之后，才有效

Parameters:	filename (str) – the path of file
Returns:	1 success 0 fail

batch_addmen(text: str) → int[source]¶

Call NWF_Batch_AddMem

往新词识别系统中添加一段待识别新词的内存,需要在运行NWF_Batch_Start()之后，才有效

Parameters:	text (str) – text string
Returns:	1 success 0 fail

batch_complete() → int[source]¶

Call NWF_Batch_Complete

新词识别添加内容结束,需要在运行NWF_Batch_Start()之后，才有效

Returns:	1 success 0 fail

batch_getresult(format_json: bool = False) → str[source]¶

Call NWF_Batch_GetResult

获取新词识别的结果, 需要在运行NWF_Batch_Complete()之后，才有效

Parameters:	format_json (bool) – get json format or not
Returns:	输出格式为

新词1】 【权重1】 【新词2】 【权重2】 ...
Json格式如下：
[
   {
      "freq" : 152,
      "pos" : "n_new",
      "weight" : 77.884208081632579,
      "word" : "公允价值"
   },
   {
      "freq" : 71,
      "pos" : "n_new",
      "weight" : 75.102183562405372,
      "word" : "长期股权投资"
   }
]

result2user_dict() → int[source]¶

Call NWF_Result2UserDict

将新词识别结果导入到用户词典中,需要在运行NLPIR_NWI_Complete()之后，才有效. 如果需要将新词结果永久保存，建议在执行NLPIR_SaveTheUsrDic

Returns:	bool, true:success, false:fail

get_last_error_msg() → str[source]¶: 对应每个组件获取异常信息的函数

nlpir.native.summary module¶

class nlpir.native.summary.Summary(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

load_mode = 1¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call DS_Init

Parameters:	data_path – encode – license_code –
Returns:

exit_lib() → bool[source]¶

Call DS_Exit

Returns:

get_last_error_msg() → str[source]¶

Call DS_GetLastErrMsg

Returns:

single_doc(text: str, sum_rate: float = 0.0, sum_len: int = 500, html_tag_remove: int = 0, sentence_count: int = 0)[source]¶

Call DS_SingleDoc

生成单文档摘要, make summarization

Parameters:

text (str) – 文档内容 text content
sum_rate (float) – 文档摘要占原文百分比(为0.00则不限制） the percentage of summarization length comparing to original text (0.00 represent no limit)
sum_len (int) – 用户限定的摘要长度(为0则不限制）The max len of summarization(0 will no limit)
html_tag_remove (bool) – 是否需要对原文进行Html标签的去除 remove the html tag or not
sentence_count (int) – 用户限定的句子数量（为0则不限制）

Returns:

摘要字符串；出错返回空串 the summarization content, get null string if occurs error.

single_doc_e(text: str, sum_rate: float = 0.0, sum_len: int = 500, html_tag_remove: int = 0, sentence_count: int = 0)[source]¶

Call DS_SingleDocE

生成单文档摘要该函数支持多线程，是多线程安全的, make summarization with threading safe

Parameters:

text (str) – 文档内容 text content
sum_rate (float) – 文档摘要占原文百分比(为0.00则不限制） the percentage of summarization length comparing to original text (0.00 represent no limit)
sum_len (int) – 用户限定的摘要长度(为0则不限制）The max len of summarization(0 will no limit)
html_tag_remove (bool) – 是否需要对原文进行Html标签的去除 remove the html tag or not
sentence_count (int) – 用户限定的句子数量（为0则不限制）

Returns:

摘要字符串；出错返回空串 the summarization content, get null string if occurs error.

file_process(text_filename: str, sum_rate: float = 0.0, sum_len: int = 500, html_tag_remove: int = 0, sentence_count: int = 0)[source]¶

Call DS_FileProcess

生成单文档摘要该函数支持多线程,是多线程安全的, make summarization from file with threading safe

Parameters:

text_filename (str) – 文档文件路径 text file path
sum_rate (float) – 文档摘要占原文百分比(为0.00则不限制） the percentage of summarization length comparing to original text (0.00 represent no limit)
sum_len (int) – 用户限定的摘要长度(为0则不限制）The max len of summarization(0 will no limit)
html_tag_remove (bool) – 是否需要对原文进行Html标签的去除 remove the html tag or not
sentence_count (int) – 用户限定的句子数量（为0则不限制）

Returns:

摘要字符串；出错返回空串 the summarization content, get null string if occurs error.

nlpir.native.key_extract module¶

class nlpir.native.key_extract.KeyExtract(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

A dynamic link library native class for Key Words Extract

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call KeyExtract_Init

Parameters:	data_path (str) – encode (int) – license_code (str) –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call KeyExtract_Exit

Returns:	exit success or not

get_keywords(line: str, max_key_limit: int = 50, format_opt: int = 0) → str[source]¶

Call KeyExtract_GetKeyWords

Extract keyword from text, 从文本中获取关键词

Parameters:	line – the input paragraph max_key_limit – maximum of key words, up to 50 format_opt – output format option, there three options: `nlpir.native.nlpir_base.OUTPUT_FORMAT_SHARP` get string split by sharp `nlpir.native.nlpir_base.OUTPUT_FORMAT_JSON` get json format `nlpir.native.nlpir_base.OUTPUT_FORMAT_EXCEL` get csv format
Returns:	the keyword with weight

Split with #:

科学发展观/n/23.80/12#宏观经济/n/12.20/12#

JSON形式:

[
    {
        'freq': 2,
        'pos': 'n_new',
        'weight': 7.771335980376418,
        'word': '国家权力'
    },{
        'freq': 7,
        'pos': 'n',
        'weight': 7.438759706600493,
        'word': '权力'
    },{
        'freq': 1,
        'pos': 'nrf',
        'weight': 5.280000338096665,
        'word': '孟德斯鸠'
    },{ ...
    }, ...
]

get_file_keywords(filename: str, max_key_limit: int = 50, format_opt: int = 0) → str[source]¶

Call KeyExtract_GetKeyWords

Extract keyword from file, 从文本文件中获取关键词

Parameters:	filename – the input text file max_key_limit – maximum of key words, up to 50 format_opt – same as `get_keywords()`
Returns:	the keyword with weight

Split with #

科学发展观/n/23.80/12#宏观经济/n/12.20/12#

JSON形式:

[
    {
        'freq': 2,
        'pos': 'n_new',
        'weight': 7.771335980376418,
        'word': '国家权力'
    },{
        'freq': 7,
        'pos': 'n',
        'weight': 7.438759706600493,
        'word': '权力'
    },{
        'freq': 1,
        'pos': 'nrf',
        'weight': 5.280000338096665,
        'word': '孟德斯鸠'
    },{ ...
    }, ...
]

import_user_dict(filename: str, overwrite: bool = False)[source]¶

Call KeyExtract_ImportUserDict

Import a user dict to the system, the format of the dict file:

word1 pos_tag
word2 pos_tag

If you import a user dict to the system, the user dict will save to the system (in Data directory). You cannot delete the word in the user dict from the system use clean_user_word() or del_usr_word().

Parameters:	filename (str) – the path of user dict file overwrite (bool) – overwrite the current user dict or not
Returns:	import success or not 1->True 2->False

add_user_word(word: str) → int[source]¶

Call KeyExtract_AddUserWord

Add a word to the user dictionary ,example:

单词 词性

or:

单词 (default n)

The added word only add in memory and will not affect the user dict, you can use clean_user_word() or del_usr_word() to delete the word or all the words in memory. If you want to save to the user dict ,use save_the_usr_dic() to save to the Data directory.

Parameters:	word (str) –
Returns:	1,true ; 0,false

clean_user_word() → int[source]¶

Call KeyExtract_CleanUserWord

Clean all temporary added user words, more info see add_user_word()

Returns:	1,true ; 0,false

clean_current_user_word() → int[source]¶

Call KeyExtract_CleanCurrentUserWord Clean all Current temporary added user words and restore previous stored data

** Now Only for win and linux x64 **

Returns:	1,true ; 0,false

save_the_usr_dic() → int[source]¶

Call KeyExtract_SaveTheUsrDic

Save in-memory dict to user dict, more info see add_user_word()

Returns:	1,true; 2,false

del_usr_word(word: str) → int[source]¶

Call KeyExtract_DelUsrWord

Delete a word from the user dictionary, more info see add_user_word()

Parameters:	word (str) – the word to be delete
Returns:	-1, the word not exist in the user dictionary; else, the handle of the word deleted

import_key_blacklist(filename: str, pos_blacklist: Optional[str] = None) → int[source]¶

Call KeyExtract_ImportKeyBlackList

Import keyword black list

This function will save words to KeyBlackList.pdat , if you want to remove the words form the system need to backup it before use this function. Or use the function nlpir.key_extract.import_blacklist() , That function will backup that file automatically and you can use nlpir.key_extract.clean_blacklist() to clean current blacklist and restore the origin file.

This list of word will not affect the key word extract and segmentation

Parameters:	filename – A word list that the words want to import to the blacklist (stop word list), 一个停用词词表,里面为想进行屏蔽的词,也可以包括别的词,是否不进行抽取是按照词表中的词性来确定的. pos_blacklist – A list of pos that want to block in the system, 想要屏蔽的词的词性
Returns:	number of words that import to the systems

batch_start() → int[source]¶

Call KeyExtract_Batch_Start

启动关键词识别

Returns:	rue:success, false:fail

batch_add_file(filename) → int[source]¶

Call KeyExtract_Batch_AddFile

往关键词识别系统中添加待识别关键词的文本文件, 需要在运行 batch_start() 之后，才有效

Parameters:	filename – 文件名
Returns:	true:success, false:fail

batch_addmen(txt: str) → bool[source]¶

Call KeyExtract_Batch_AddMem

往关键词识别系统中添加一段待识别关键词的内存,需要在运行 batch_start() 之后，才有效

Parameters:	txt – 文件名
Returns:	true:success, false:fail

batch_complete() → int[source]¶

Call KeyExtract_Batch_Complete

关键词识别添加内容结束,需要在运行 batch_start() 之后，才有效

Returns:	true:success, false:fail

batch_getresult(weight_out: bool) → str[source]¶

Call KeyExtract_Batch_GetResult

获取关键词识别的结果,需要在运行 batch_complete() 之后，才有效

Parameters:	weight_out – 是否需要输出每个关键词的权重参数
Returns:	输出格式为【关键词1】【权重1】【关键词2】【权重2】 …

get_last_error_msg() → str[source]¶

Call KeyExtract_GetLastErrorMsg

Returns:	error message

nlpir.native.deep_classifier module¶

class nlpir.native.deep_classifier.DeepClassifier(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

A dynamic link library native class for Classify using deep learning

FEATURE_COUNT = 800¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call DeepClassifier_Init

Init DeepClassifier

Parameters:	data_path – encode – license_code –
Returns:

exit_lib() → bool[source]¶

Call DeepClassifier_Init

Returns:

get_last_error_msg() → str[source]¶

Call DeepClassifier_GetLastErrorMsg

Returns:

new_instance(feature_count: int) → int[source]¶

Call DeepClassifier_NewInstance

New a DeepClassifier Instance. This function must be invoked before classify, and need be deleted when exit the process. Delete instance can use the function delete_instance()

Parameters:	feature_count – Feature count
Returns:	DeepClassifier Handle if success; otherwise return -1;

delete_instance(instance: int) → int[source]¶

Call DeepClassifier_DeleteInstance

Delete a DeepClassifier Instance with handle. The function must be invoked before release a specific classifier. The instance can be retrieve by new_instance()

Parameters:	instance – DeepClassifier Handle
Returns:

add_train(classname: str, text: str, handler: int = 0) → bool[source]¶

Call DeepClassifier_AddTrain

DeepClassifier add train dataset on given text in Memory

Parameters:	classname – class name text – text content handler – classifier handler
Returns:	add success or not

add_train_file(classname: str, filename: str, handler: int = 0) → int[source]¶

Call DeepClassifier_AddTrainFile

DeepClassifier add train dataset on given text in file

Parameters:	classname – class name filename – text file name handler – classifier handler
Returns:	success or fail

train(handler: int = 0) → int[source]¶

Call DeepClassifier_Train

DeepClassifier Training on given text in Memory. After training, the training result will stored. Then the classifier can load it with load_train_result() (offline or online).

Parameters:	handler – classifier handler
Returns:	success or not

load_train_result(handler: int = 0) → int[source]¶

Call DeepClassifier_LoadTrainResult

DeepClassifier Load already training data

Parameters:	handler – classifier handler
Returns:	success or not

export_features(filename: str, handler: int = 0) → int[source]¶

Call DeepClassifier_ExportFeatures

DeepClassifier Exports Features after training

Parameters:	filename – save path handler – classifier handler
Returns:	success or not

classify(text: str, handler: int = 0) → str[source]¶

Call DeepClassifier_Classify

DeepClassifier Classify on given text in Memory

Parameters:	text – text handler – classifier handler
Returns:	classify result , a class name

classify_ex(text: str, handler: nlpir.native.deep_classifier.LP_c_int = 0)[source]¶

Call DeepClassifier_ClassifyEx

DeepClassifier Classify on given text in Memory, return multiple class with weights, sorted by weights

Parameters:	text – text handler – classifier handler
Returns:	result with weight, For instance: `政治/1.20##经济/1.10,` `bookyzjs/7.00##bookxkfl/6.00##booktslx/5.00##bookny-xyfl/4.00##`

classify_file(filename: str, handler: int = 0)[source]¶

Call DeepClassifier_ClassifyFile

DeepClassifier Classify on given text in file

Parameters:	filename – file name of text handler – classifier handler
Returns:	result same as `classify()`

classify_file_ex(filename: str, handler: int = 0)[source]¶

Call DeepClassifier_ClassifyExFile

DeepClassifier Classify on given text in file

Parameters:	filename – file name of text handler – classifier handler
Returns:	result same as `classify_ex()`

nlpir.native.classifier module¶

class nlpir.native.classifier.StDoc[source]¶: Bases: _ctypes.Structure

class nlpir.native.classifier.Classifier(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call classifier_init

Parameters:	data_path – encode – license_code –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call classifier_exit

Returns:	exit success or not

get_last_error_msg() → str[source]¶: 对应每个组件获取异常信息的函数

exec_1(data: nlpir.native.classifier.StDoc, out_type: int = 0)[source]¶

Call classifier_exec1

对输入的文章结构进行分类

Parameters:	data – 文章结构 out_type – 输出是否包括置信度, 0 没有置信度 1 有置信度
Returns:	主题类别串各类之间用隔开，类名按照置信度从高到低排序举例：“要闻敏感诉讼”, “要闻 1.00 敏感 0.95 诉讼 0.82”

exec(title: str, content: str, out_type: int)[source]¶

Call classifier_exec

对输入的文章进行分类

Parameters:	title – 文章标题 content – 文章内容 out_type – 输出知否包括置信度,同 `exec_1()`
Returns:	同 `exec_1()`

exec_file(filename: str, out_type: int) → str[source]¶

Call classifier_execFile

Parameters:	filename – 文件名 out_type – 输出是否包括置信度, 0 没有置信度 1 有置信度
Returns:	主题类别串各类之间用隔开，类名按照置信度从高到低排序举例：“要闻敏感诉讼”, “要闻 1.00 敏感 0.95 诉讼 0.82”

detail(class_name: str)[source]¶

Call classifier_detail

对于当前文档，输入类名，取得结果明细

Parameters:	class_name – 结果类名
Returns:	结果明细例如:

RULE3:
SUBRULE1: 内幕 1
SUBRULE2: 股市 1      基金 3    股票 8
SUBRULE3: 书摘 2

set_sim_thresh(sim: float)[source]¶

Call classifier_setsimthresh

设置阈值

Parameters:	sim – 阈值
Returns:

nlpir.native.sentiment module¶

class nlpir.native.sentiment.SentimentNew(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call ST_Init

Parameters:	data_path – encode – license_code –
Returns:

exit_lib() → bool[source]¶

Call ST_Exit

Returns:

get_last_error_msg() → str[source]¶

Call ST_GetLastErrorMsg

Returns:

get_one_object_result(title: str, content: str, analysis_object: str) → str[source]¶

Call ST_GetOneObjectResult

Parameters:	title – content – analysis_object –
Returns:

get_multi_object_result(title: str, content: str, object_rule_file: str) → str[source]¶

Call ST_GetMultiObjectResult

Parameters:	title – content – object_rule_file – see Appendix II: Multiple Object configure sample
Returns:

get_sentence_point(sentence: str) → str[source]¶

Call ST_GetSentencePoint

Get multiple object sentimental result

Parameters:	sentence –
Returns:	double,Sentimental point

get_sentiment_point(sentence: str) → float[source]¶

Call ST_GetSentimentPoint

Get multiple object sentimental result

Parameters:	sentence –
Returns:	double,Sentimental point

import_user_dict(filename: str, over_write: bool = False) → int[source]¶

Call ST_ImportUserDict

Import User-defined dictionary, same as nlpir.native.ictclas.ICTCLAS.import_user_dict()

Parameters:	filename – over_write –
Returns:

process_dir(path: str) → str[source]¶

Call ST_ProcesDir

批量处理指定的目录下的文本文件. 分析结果, 输出到指定的Excel文件中

Parameters:	path –
Returns:	path目录下, 自动生成 `SentimentRankResult.xls`,返回该文件的全路径名称

class nlpir.native.sentiment.SentimentAnalysis(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

EMOTION_HAPPY = 0¶

EMOTION_GOOD = 1¶

EMOTION_ANGER = 2¶

EMOTION_SORROW = 3¶

EMOTION_FEAR = 4¶

EMOTION_EVIL = 5¶

EMOTION_SURPRISE = 6¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call LJST_Init

Parameters:	data_path – encode – license_code –
Returns:

exit_lib() → bool[source]¶

Call LJST_Exits

Returns:

get_last_error_msg() → str[source]¶

Returns:

get_paragraph_sent(paragraph: str) → Tuple[bool, str][source]¶

Call LJST_GetParagraphSent

Get sentiment analyze result

Parameters:	paragraph –
Returns:

get_file_sent(filename: str) → Tuple[bool, str][source]¶

Call LJST_GetFileSent

Get sentiment analyze result

Parameters:	filename –
Returns:

import_user_dict(filename: str, over_write: bool = False)[source]¶

Call LJST_ImportUserDict

Import User-defined dictionary, same as nlpir.native.ictclas.ICTCLAS.import_user_dict()

Parameters:	filename – over_write –
Returns:

get_paragraph_sent_e(paragraph: str) → str[source]¶

Call LJST_GetParagraphSentE

Get sentiment analyze result

Parameters:	paragraph –
Returns:

get_file_sent_e(filename: str) → str[source]¶

Call LJST_GetFileSentE

Get sentiment analyze result

Parameters:	filename –
Returns:

nlpir.native.key_scanner module¶

nlpir.native.key_scanner.ENCODING_UTF8_FJ = 5¶: UTF8编码转换过程中自动繁简转换处理，扫描过滤功能建议使用

nlpir.native.key_scanner.SCAN_MODE_NORMAL = 0¶: 正常扫描模式

nlpir.native.key_scanner.SCAN_MODE_SHAPE = 1¶: 形变扫描模式

nlpir.native.key_scanner.SCAN_MODE_PINYIN = 2¶: 拼音扫描模式

nlpir.native.key_scanner.SCAN_MODE_CHECK = 3¶: 校对扫描模式

class nlpir.native.key_scanner.KeyScanner(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

A dynamic link library native class for Keyword Scan

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call KS_Init

Parameters:	data_path (str) – encode (int) – license_code (str) –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call KS_Exit

Returns:	exit success or not

get_last_error_msg() → str[source]¶

Call KS_GetLastErrorMsg

Returns:	error message

new_instance(filter_type_index: int = 0) → int[source]¶

Call KS_NewInstance

Get a instance from system for executing other functions. The function must be invoked before multiple keyword scan filter. This function will alloc memory , it need to be free memory by using delete_instance() after finish all executions from this handle.

Parameters:	filter_type_index – which No of filter want to be used in this instance. The filter file will save into Data/KeyScanner/filter{no}*
Returns:	a handle from system if success; otherwise return -1;

delete_instance(handle: int) → int[source]¶

Call KS_DeleteInstance

Delete handle created by :func`new_instance`. Once delete handle from system, this handle cannot be used in any situation or will invoke critical errors.

Parameters:	handle – the handle want to be deleted
Returns:	success or not

import_user_dict(filename: str, over_write: bool = False, pinyin_abbrev_needed: bool = False, handle=0) → int[source]¶

Call ImportUserDict

Import User-defined dictionary 导入用户词典, 此操作为全局操作会影响其他 instance 的过滤

文本文件每行的格式为: 词条词类权重 (注意，最多定义255个类别), 例:

AV电影 色情 2
六合彩 涉赌 8 1

复杂过滤条件: 支持与或非处理 ;表示或关系，+表示与关系，-表示否格式如下:

{key11;key12;key13;...;key1N}+{key21;key22;key23;...;key2N}+...+{keyM1;keyM2;keyM3;...;keyMN}-{keyN}

示例:

{中国;中华;中华人民共和国;中国共产党;中共}+{伟大;光荣;正确}-{中华民国;国民党}  政治类 5

表示的是文本内容中包含 中国;中华;中华人民共和国;中国共产党;中共 中的一种，同时出现 伟大;光荣;正确 中的一个，但不能出现 中华民国;国民党 的任何一个

Parameters:	filename – path of user dictionary pinyin_abbrev_needed – over_write – true将覆盖系统已经有的词表；否则将采用追加的方式追加不良词表 handle – handle of KeyScanner
Returns:	success or not

delete_user_dic(text: str, handle: int) → int[source]¶

Call DeleteUserDict

Delete User-defined dictionary 删除用户词典, 此操作为全局操作, 会删除词典文件并影响所有 instance

文本文件每行的格式为: 词条 , 例如:

AV电影
习近平

Parameters:	text – Text of user dictionary handle – handle of KeyScanner
Returns:	The number of lexical entry deleted successfully 成功删除的词典条数

delete_user_dic_from_file(filename: str, handle: int) → int[source]¶

Call DeleteUserDict

Delete User-defined dictionary 删除用户词典, 此操作为全局操作, 会删除词典文件并影响所有 instance

文本文件每行的格式为: 词条 , 例如:

AV电影
习近平

Parameters:	filename – Text filename for user dictionary handle – handle of KeyScanner
Returns:	The number of lexical entry deleted successfully 成功删除的词典条数

scan(content: str, handle: int = 0) → str[source]¶

Call KS_Scan

扫描输入的文本内容

Parameters:	content – 文本内容 handle – handle of KeyScanner
Returns:	涉及不良的所有类别与权重，按照权重排序。如: `色情/10#暴力/1#` , `政治反动/2#FLG/1#涉领导人/1#` , `""` : 表示无扫描命中结果

scan_detail(content: str, scan_mode: int = 0, handle: int = 0) → str[source]¶

Call KS_ScanDetail

扫描输入的文本内容,获得详细结果

Parameters:	scan_mode – 扫描模式 content – 文本内容 handle – handle of KeyScanner
Returns:	返回包含了扫描结果的内容，扫描结果明细:

{
    "Details": ["chou傻逼xi禁评"],
    "Rules": ["傻逼","xi禁评"],
    "filename": "",
    "illegal" :{
        "classes":[
            {
                "freq":1,
                "word":"粗言秽语"
            },{
                "freq":1,
                "word":"污言秽语"
            },{
                "freq":1,
                "word":"新华社禁用"
            },{
                "freq":1,"word":"一号首长"
            }
        ],
        "hit_count":4,
        "keys":["傻逼","xi禁评"],
        "scan_val":13.333333333333332
    },
    "legal": {
        "hit_count":0,
        "scan_val":0.0
    },
    "line_id":0,
    "org_file":"",
    "score":13.333333333333332
}

scan_file(filename: str, handle: int = 0) → str[source]¶

Call KS_ScanFile

扫描输入的文本文件内容

Parameters:	filename – 文本文件名 handle – handle of KeyScanner
Returns:	same as `scan()`

scan_file_detail(filename: str, handle: int = 0) → str[source]¶

Call KS_ScanFileDetail

扫描输入的文本文件内容

Parameters:	filename – 文本文件名 handle – handle of KeyScanner
Returns:	same as `scan_detail()`

scan_line(filename: str, result_filename: str, handle: int = 0, encrypt: int = 0, scan_mode: int = 0) → int[source]¶

Call KS_ScanLine

按行扫描输入的文本文件内容

Parameters:	filename – 输入的文本文件名 result_filename – 输出的结果文件名 handle – handle of KeyScanner encrypt – 0 不加密；1，加密 scan_mode –
Returns:	same as `scan_detail()`

scan_stat(result_file, handle: int = 0) → int[source]¶

Call KS_ScanStat

输出扫描结果的命中统计报告，利于进一步的分析核查

Parameters:	result_file – 输出结果的文件文件 handle – handle of KeyScanner
Returns:	成功扫描到问题的文件数

scan_dir(input_dir_path: str, result_path: str, filter: str, thread_count: int = 10, encrypt: bool = False, scan_mode: int = 0) → int[source]¶

Call KS_ScanDir

多线程扫描按行扫描输入的文本夹文件内容

Parameters:	input_dir_path – 输入的文件夹路径 result_path – 输出结果的文件夹路径 filter – 输入的文件后缀名 thread_count – 线程数，默认10个 encrypt – 0 不加密；1，加密 scan_mode –
Returns:	成功扫描到问题的文件数

merge_result(path: str) → None[source]¶

Merge多线程的扫描结果

Parameters:	path –
Returns:

scan_add_stat(result_file: str, handle: int) → int[source]¶

将handle线程扫描结果归并到0线程

Parameters:	result_file – handle –
Returns:

stat_result_filter(input_filename: str, result_filename: str, threshold: float = 5.0) → int[source]¶

Call KS_StatResultFilter

对扫描的统计结果进行过滤分析

Parameters:	input_filename – 输入的结果文件名 result_filename – 输出结果的文件名 threshold – 不良得分的阈值
Returns:	成功扫描到问题的文件数

scan_result_filter(input_filename: str, result_filename: str, threshold: float = 9.0) → int[source]¶

Call KS_ScanResultFilter

对扫描的详细结果文件进行过滤分析

Parameters:	input_filename – 输入的结果文件名 result_filename – 输出结果的文件名 threshold – 不良得分的阈值
Returns:	成功扫描到问题的文件数

decrypt(input_dir_path: str, result_path: str) → int[source]¶

Call KS_Decrypt

多线程转换扫描结果

Parameters:	input_dir_path – 输入的文件夹路径 result_path – 输出结果的文件夹路径
Returns:

export_dict(filename: str, handle: int = 0) → int[source]¶

Call KS_ExportDict

ExportDict dictionary 导出已经定义的不良词词典, 为保护知识产权，该功能仅局限于管理员内部调度使用

文本文件的格式为: 词条词类权重 (注意，最多定义255个类别) 例如:

AV电影 色情 2
六合彩 涉赌 8 1

Parameters:	filename – Text filename for user dictionary handle – handle of KeyScanner
Returns:	The number of lexical entry imported successfully 成功导入的词典条数

nlpir.native.cluster module¶

class nlpir.native.cluster.Cluster(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

load_mode = 1¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call CLUS_init

Parameters:	data_path – encode – license_code –
Returns:	1 success Other fail

exit_lib() → bool[source]¶

Call CLUS_exit

Returns:	exit success or not

get_last_error_msg() → str[source]¶

Call CLUS_GetLastErrorMsg

Returns:

set_parameter(max_clus: int, max_doc: int) → bool[source]¶

Call CLUS_SetParameter

设置最大类别数以及最大输入文档数,类和类内的文档均已按照重要性和及时性排过序

Parameters:	max_clus – 最大类别数 max_doc – 最大文档数
Returns:	是否成功

add_content(text: str, signature: str) → bool[source]¶

Call CLUS_AddContent

追加内存内容,在进程中此函数可以在打印结果之前执行多次

Parameters:	text – 正文 signature – 唯一标识
Returns:	是否成功

add_file(filename: str)[source]¶

Call CLUS_AddFile

追加文件内容,在进程中此函数可以在打印结果之前执行多次

Parameters:	filename – 正文文件
Returns:	是否成功

get_latest_result(xml_filename: str, result_path: Optional[str] = None) → Tuple[bool, str][source]¶

Call CLUS_GetLatestResult

输出结果到xml文件中

<?xml version="1.0" encoding="gb2312" standalone="yes" ?>
<LJCluster-Result>
<clusnum>2</clusnum>

<clus id="0">
    <feature>奥巴马 竞选 财务部</feature>
    <docs num="6">
       <doc>2</doc>
       <doc>3</doc>
       <doc>35</doc>
       <doc>86</doc>
       <doc>345</doc>
       <doc>975</doc>
    </docs>
</clus>

<clus id="1">
    <feature>林志玲 影视 电影 广告</feature>
    <docs num="4">
       <doc>45</doc>
       <doc>86</doc>
       <doc>135</doc>
       <doc>286</doc>
    </docs>
</clus>
</LJCluster-Result>

Parameters:	xml_filename – 输出文件名 result_path – 输出路径, 按照聚类结果作为不同子目录存储
Returns:	是否成功

get_latest_result_e(result_path: Optional[str] = None) → str[source]¶

Call CLUS_GetLatestResultE

输出xml结果到内存

Parameters:	result_path –
Returns:	xml like `get_latest_result()`

clean_data() → None[source]¶

Call CLUS_CleanData

清空历史数据

Returns:

nlpir.native.doc_extractor module¶

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_PERSON = 0¶: 人名

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_LOCATION = 1¶: 地名

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_ORGANIZATION = 2¶: 机构名

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_KEYWORD = 3¶: 关键词

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_AUTHOR = 4¶: 文章作者

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_MEDIA = 5¶: 媒体

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_COUNTRY = 6¶: 文章对应的所在国别

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_PROVINCE = 7¶: 文章对应的所在省份

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_ABSTRACT = 8¶: 文章的摘要

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_POSITIVE = 9¶: 文章的正面情感词

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_NEGATIVE = 10¶: 文章的负面情感词

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_TEXT = 11¶: 文章去除网页等标签后的正文

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_TIME = 12¶: 时间词

nlpir.native.doc_extractor.DOC_EXTRACT_TYPE_USER = 13¶: 用户自定义的词类，第一个自定义词后续的自定义词，依次序号为：DOC_EXTRACT_TYPE_USER + 1 , DOC_EXTRACT_TYPE_USER + 2 , …

nlpir.native.doc_extractor.HTML_REMOVER_REQUIRED = 32768¶: 是否需要去除网页标签的功能选项

class nlpir.native.doc_extractor.DocExtractor(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

A dynamic link library native class for Document Extractor

DOC_EXTRACT_DELIMITER = '#'¶: 分隔符

DOC_EXTRACT_TYPE_MAX_LENGTH = 600¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call DE_Init

Parameters:	data_path (str) – encode (int) – license_code (str) –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call DE_Exit

Returns:	exit success or not

get_last_error_msg() → str[source]¶

Call DE_GetLastErrorMsg

Returns:	error message

pares_doc_e(text: str, user_def_pos: str, summary_needed: bool = True, func_required: int = 65535) → int[source]¶

Call DE_ParseDocE

生成单文档摘要

Parameters:	text – 文档内容 user_def_pos – 用户自定义的词性标记, 最多三种（人名、地名、机构名、媒体等内置，无需设置, 不同词类之间采用#分割, 如 `gms#gjtgj#g` summary_needed – 是否需要计算摘要 func_required –
Returns:	用于获取内容的handle, 获取内容完毕后应使用 `release_handle()` 释放对应资源

release_handle(handle: int) → None[source]¶

Call DE_ReleaseHandle

释放 parse_doc_e() 结果所占据的空间

Parameters:	handle – `parse_doc_e()` 执行后返回的HANDLE
Returns:

get_result(handle: int, doc_extract_type: int) → str[source]¶

Call DE_GetResult

从运行完的 parse_doc_e() 结果中，获取指定抽取的结果内容

Parameters:	handle – `parse_doc_e()` 执行后返回的HANDLE doc_extract_type – 获取的抽取类型，从DOC_EXTRACT_TYPE_PERSON开始的结果
Returns:

get_sentiment_score(handle: int) → int[source]¶

Call DE_GetSentimentScore

从运行完的 parse_doc_e() 结果中，获取指文章的情感得分

Parameters:	handle – `parse_doc_e()` 执行后返回的HANDLE
Returns:	情感正负得分

compute_sentiment_doc(text: str) → int[source]¶

Call DE_ComputeSentimentDoc

生成单文档情感分析结果

Parameters:	text – 文档内容
Returns:

import_sentiment_dict(filename: str) → int[source]¶

Call DE_ImportSentimentDict

导入用户自定义的情感词表，每行一个词，空格后加上正负权重，如: 语焉不详 -2

若导入的情感词属于新词, 需先在用户词典中导入, 否则情感识别自动跳跃

Parameters:	filename –
Returns:

import_user_dict(filename: str, overwrite: bool = False) → int[source]¶

Call DE_ImportUserDict

导入用户词典, see nlpir.native.ictclas.ICTCLAS.import_user_dict()

Parameters:	filename – overwrite –
Returns:

add_user_word(word: str) → int[source]¶

Call DE_AddUserWord

Add a word to the user dictionary, see nlpir.native.ictclas.ICTCLAS.add_user_word()

Parameters:	word –
Returns:

clean_user_word() → int[source]¶

Call DE_CleanUserWord

Clean all temporary added user words, see nlpir.native.ictclas.ICTCLAS.clean_user_word()

Returns:

save_the_usr_dic() → int[source]¶

Call DE_SaveTheUsrDic

Save in-memory dict to user dict, see nlpir.native.ictclas.ICTCLAS.save_the_usr_dic() :return:

del_usr_word(word: str) → int[source]¶

Call DE_DelUsrWord

Delete a word from the user dictionary, see nlpir.native.ictclas.ICTCLAS.del_usr_word()

Parameters:	word –
Returns:

import_key_blacklist(filename: str, pos_blacklist: str) → int[source]¶

Call DE_ImportKeyBlackList

Import keyword black list, see nlpir.native.key_extract.KeyExtract.import_key_blacklist()

Parameters:	filename – pos_blacklist –
Returns:

nlpir.native.text_similarity module¶

nlpir.native.text_similarity.SIM_MODEL_CHAR = 1¶: 字模型，速度最快，适用于相对规范的短文本

nlpir.native.text_similarity.SIM_MODEL_WORD = 2¶: 词模型，速度适中，常规适用于正常规范的长文档

nlpir.native.text_similarity.SIM_MODEL_KEY = 3¶: 主题词模型，速度最慢，考虑语义最多，适合于复杂文本

class nlpir.native.text_similarity.TextSimilarity(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call TS_Init

Parameters:	data_path – encode – license_code –
Returns:

exit_lib() → bool[source]¶

Call DS_Exit

Returns:

get_last_error_msg() → str[source]¶

Call TS_GetLastErrorMsg

Returns:

compute_sim(text_1: str, text_2: str, model: int = 2) → float[source]¶

Call TS_ComputeSim

Parameters:	text_1 – text_2 – model –
Returns:

compute_sim_file(filename_1: str, filename_2: str, model: int = 2) → float[source]¶

Call TS_ComputeSimFile

Parameters:	filename_1 – filename_2 – model –
Returns:

nlpir.native.eye_checker module¶

class nlpir.native.eye_checker.EyeChecker(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: nlpir.native.nlpir_base.NLPIRBase

TODO report_type or doc_type

A dynamic link library native class for 09 Eys Checker

DOC_EXTRACT_DELIMITER = '#'¶: 分隔符

DOC_EXTRACT_TYPE_MAX_LENGTH = 600¶

load_mode = 1¶

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

Call NERICS_Init

Parameters:	data_path (str) – encode (int) – license_code (str) –
Returns:	1 success 0 fail

exit_lib() → bool[source]¶

Call NERICS_Exit

Returns:	exit success or not

get_last_error_msg() → str[source]¶

Call DE_GetLastErrorMsg

Returns:	error message

import_field_dict(field_dict_file: str, pinyin_abbrev_needed: bool = False, overwrite: bool = True) → int[source]¶

Import field dictionary

Parameters:	field_dict_file – pinyin_abbrev_needed – overwrite –
Returns:

new_instance() → int[source]¶

Description: New a NERICS Instance

The function must be invoked before mulitiple keyword scan filter

Parameters : Returns : NERICS_HANDLE: KeyScan Handle if success; otherwise return -1; Author : Kevin Zhang History :

1.create 2016-11-15

return:

delete_instance(handle: int) → int[source]¶

Parameters:	handle –
Returns:

import_doc(report_file: str, url_prefix: str = '', handle: int = 0) → str[source]¶

Func Name : NERICS_ImportDoc

Description: Read a Report file and save the result in file with XML format

Parameters : sReportFile: Report File

sURLPrefix: URL前缀路径 handle: NERICS handle, generated by NERICS_NewInstance

Returns : Return result file name: sXMLFile: XML file stored Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	report_file – url_prefix – handle –
Returns:

load_doc_result(result_xml_file: str, handle: int = 0) → int[source]¶

Func Name : NERICS_LoadDocResult

Description: Read a result XML file and save the result in file with XML format

Parameters : sReportFile: Report File

sURLPrefix: URL前缀路径 handle: NERICS handle, generated by NERICS_NewInstance

Returns : Return result file name: sXMLFile: XML file stored Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	result_xml_file – handle –
Returns:

check_report_f(report_file: str, url_prefix: str = '', organization: str = '', report_type: int = 0, format_opt: int = 1, handle: str = 0) → str[source]¶

Func Name : NERICS_CheckReportF

Description: Check a Report file and save the result in file with XML format

Parameters : sReportFile: Report File: 支持doc,docx,xml文件: sURLPrefix: URL前缀路径 nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance int nResultFormat：0: XML; 1:Jason

Returns : Return result file name: sXMLFile: XML file stored

Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	report_file – url_prefix – organization – report_type – format_opt – handle –
Returns:

check_report_m(report_text: str, url_prefix: str = '', organization: str = '', report_type: int = 0, format_opt: int = 1, handle: str = 0) → str[source]¶

Func Name : NERICS_CheckReportM

Description: Check a Report text memory and save the result in file with XML format

Parameters : sReportFile: Report File: 支持doc,docx,xml文件: sURLPrefix: URL前缀路径 nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance int nResultFormat：0: XML; 1:Jason

Returns : Return result file name: sXMLFile: XML file stored

Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	report_text – url_prefix – organization – report_type – format_opt – handle –
Returns:

extract_knowledge(report_text: str, report_type: int = 0) → str[source]¶

Func Name : NERICS_ExtractKnowledge

Description: Extract Knowledge from a text， given a configure string with XML format nType: Report Type, Default is RPT_UNSPECIFIC

Parameters : sReportFile: Report File: 支持doc,docx,xml文件: sURLPrefix: URL前缀路径 nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance int nResultFormat：0: XML; 1:Jason

Returns : Return result file name: sXMLFile: XML file stored

Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	report_text – report_type –
Returns:

get_result(result_type: int, handle: int = 0) → str[source]¶

Func Name : NERICS_GetResult

Description: 获取分析结果，默认为JSON格式

Parameters : result_type:: handle: NERICS handle, generated by NERICS_NewInstance

Returns : Return result file name: sXMLFile: XML file stored

Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	result_type – handle –
Returns:

add_audit_rule(audit_rule: str, report_type: int = 0) → int[source]¶

Func Name : NERICS_AddAuditRule

Description: Add Audit Rule

Parameters : sAuditRule: Audit rule,需要遵循KGB Audit语法规则: nType: Report Type, Default is RPT_UNSPECIFIC

Returns : int: 1: success, other: failed. Get error message via NERICS_GetLastErrorMsg()

Author : Kevin Zhang History :

1.create 2018-9-19

Parameters:	audit_rule – report_type –
Returns:

check_report_dir(report_dir: str, organization: str, report_type: int = 0, format_opt: int = 1, thread_count: int = 10) → str[source]¶

Func Name : NERICS_CheckReportDir

Description: Scan a dir and Check all doc files

Parameters : sReportDir: Report File Directory

nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance

Returns : Return result file name: sXMLFile: XML file stored Author : Kevin Zhang History :

1.create 2018-6-5

Parameters:	report_dir – organization – report_type – format_opt – thread_count –
Returns:

revise_report_f(revise_xml_file: str, handle: int = 0) → str[source]¶

Func Name : NERICS_ReviseReportF

Description: Revised a Report file: and revised information stored in file
Parameters : sReviseXMLFile: Revised information file with XML format: nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance

Returns : Return : new docx file name with path； return “” if failed!

Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	revise_xml_file – handle –
Returns:

show_html_error(revise_xml_file: str, handle: int = 0) → str[source]¶

Description: Revised a Report file

and revised information stored in file

Parameters : sReviseXMLFile: Revised information file with XML format: nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance

Returns : Return : new docx file name with path； return “” if failed!

Author : Kevin Zhang History :

1.create 2018-5-4

Parameters:	revise_xml_file – handle –
Returns:

import_template(template_file: str, report_type: int = 0, org: str = '', area: str = '', argument: str = '') → int[source]¶

Func Name : NERICS_ImportTemplate

Description: Import a document Template

Parameters : sTemplateFile: Template file using doc or docx format: nType: document type sOrg: organization sArgumemt: arguments
Returns : Return status: int: 1: success

Author : Kevin Zhang History :

1.create 2018-5-8

2.modified in 2018-11-20

Parameters:	template_file – report_type – org – area – argument –
Returns:

edit_template(template_id: int, template_file: str, report_type: int = 0, org: str = '', area: str = '', argument: str = '') → int[source]¶

Func Name : NERICS_EditTemplate

Description: Edit a document Template

Parameters : sTemplateFile: Template file using doc or docx format: nType: document type sOrg: organization sArgumemt: arguments
Returns : Return status: int: 1: success

Author : Kevin Zhang History :

1.create 2018-5-8

2.modified in 2018-11-20

Parameters:	template_id – template_file – report_type – argument – area – org –
Returns:

find_template(report_type: int = 0, org: str = '', area: str = '', argument: str = '') → int[source]¶

Func Name : NERICS_FindTemplate

Description: Find a document Template

Parameters :: nType: document type sOrg: organization sArgumemt: arguments
Returns : Return status: int: 1: success

Author : Kevin Zhang History :

1.create 2018-5-8

Parameters:	report_type – org – area – argument –
Returns:

delete_template(template_id: int) → int[source]¶

Func Name : NERICS_DeleteTemplate

Description: delete a document Template

Parameters : nTempID: template ID Returns : Return : int

Author : Kevin Zhang History :

1.create 2018-11-20

Parameters:	template_id –
Returns:

get_template(template_id: int) → str[source]¶

Func Name : NERICS_GetTemplate

Description: Get a document Template

Parameters : nTempID: template ID Returns : Return status: const char* :template data

Author : Kevin Zhang History :

1.create 2018-11-20

Parameters:	template_id –
Returns:

get_template_count(template_id: int) → str[source]¶

Func Name : NERICS_GetTemplateCount

Description: Get document Template count

Parameters : nTempID: template ID Returns : Return status: const char* :template data

Author : Kevin Zhang History :

1.create 2018-11-20

Parameters:	template_id –
Returns:

get_current_template_info(handle: int = 0) → str[source]¶

Func Name : NERICS_GetCurTemplateInfo

Description: Get current document Template information

Parameters : Returns : Return status: const char* :template information using Jason format

Author : Kevin Zhang History :

1.create 2018-12-5

Parameters:	handle –
Returns:

get_template_list(doc_type: int, organization: str) → ctypes.c_char_p[source]¶

Func Name : NERICS_GetTemplateList

Description: Get Template information

Parameters : docType: docType;: sOrgnization: organization name

Returns : Return status: const char* :template information using Jason format

Author : Kevin Zhang History :

1.create 2018-12-5

Parameters:	doc_type – organization –
Returns:

re_check_format(check_xml: str, template_id: int, format_opt: int = 1, handle: int = 0) → str[source]¶

Func Name : NERICS_ReCheckFormat

Description: ReCheck a format

Parameters : sReportFile: Report File: 支持doc,docx,xml文件: sURLPrefix: URL前缀路径 nType: Report Type, Default is RPT_UNSPECIFIC handle: NERICS handle, generated by NERICS_NewInstance int nResultFormat：0: XML; 1:Jason

Returns : Return result file name: sXMLFile: XML file stored

Author : Kevin Zhang History :

1.create 2018-11-27

Parameters:	check_xml – template_id – format_opt – handle –
Returns:

import_kgb_rules(rule_file: str, overwrite: bool = False, report_type: int = 0) → ctypes.c_int[source]¶

Func Name : NERICS_ImportKGBRules

Description: 针对报告类型nType导入相应的KGB规则集合

Parameters : sTemplateFile: Template file using doc or docx format Returns : Return status: int

1: success

Author : Kevin Zhang History :

1.create 2018-5-8

Parameters:	rule_file – overwrite – report_type –
Returns:

import_kgb_rules_from_mem(rule_text: str, overwrite: bool = False, report_type: int = 0) → int[source]¶

Func Name : NERICS_ImportKGBRulesFromMem

Description: 针对报告类型nType导入相应的KGB规则集合

Parameters : sTemplateFile: Template file using doc or docx format Returns : Return status: int

1: success

Author : Kevin Zhang History :

1.create 2018-5-8

:param rule_text :param overwrite: :param report_type: :return:

import_error_msg(error_list_file: str) → int[source]¶

Func Name : NERICS_ImportErrorMsg

Description: Import a error message table

Parameters : sErrorListFile: Template file using doc or docx format Returns : Return status: int

1: success

Author : Kevin Zhang History :

1.create 2018-5-8

Parameters:	error_list_file –
Returns:

import_sim_dict(sim_dict_file: str) → ctypes.c_int[source]¶

Func Name : NERICS_ImportSimDict

Description: Import simary dictionary

Parameters : sErrorListFile: Template file using doc or docx format Returns : Return status: int

1: success

Author : Kevin Zhang History :

1.create 2018-5-8

Parameters:	sim_dict_file –
Returns:

import_spell_error_dict(spell_error_dict: str) → int[source]¶

Func Name : NERICS_ImportSpellErrorDict

Description: Import Spelling Error dictionary

Parameters : sSpellErrorDict: Spelling Error dictionary Returns : Return status: int

1: success

Author : Kevin Zhang History :

1.create 2018-5-8

Parameters:	spell_error_dict –
Returns:

import_user_dict(user_dict: str)[source]¶

Func Name : NERICS_ImportUserDict

Description: Import Spelling Error dictionary

Parameters : sUserDict: User defined dictionary Returns : Return status: int

1: success

Author : Kevin Zhang History :

1.create 2019-12-3

Parameters:	user_dict –
Returns:

nlpir.native.nlpir_base module¶

nlpir.native.nlpir_base.UNKNOWN_CODE = -1¶: 如果是各种编码混合，设置为-1，系统自动检测，并内部转换。会多耗费时间，不推荐使用

nlpir.native.nlpir_base.GBK_CODE = 0¶: 默认支持GBK编码

nlpir.native.nlpir_base.UTF8_CODE = 1¶: UTF8编码

nlpir.native.nlpir_base.BIG5_CODE = 2¶: BIG5编码

nlpir.native.nlpir_base.GBK_FANTI_CODE = 3¶: GBK编码，里面包含繁体字

nlpir.native.nlpir_base.UTF8_FANTI_CODE = 4¶: UTF8编码

nlpir.native.nlpir_base.OUTPUT_FORMAT_XML = 0¶: EyeChecker 使用的

nlpir.native.nlpir_base.OUTPUT_FORMAT_SHARP = 0¶: 正常的字符串按照#链接的输出新词结果

nlpir.native.nlpir_base.OUTPUT_FORMAT_JSON = 1¶: 正常的JSON字符串输出新词结果

nlpir.native.nlpir_base.OUTPUT_FORMAT_EXCEL = 2¶: 正常的CSV字符串输出新词结果,保存为csv格式即可采用Excel打开

class nlpir.native.nlpir_base.NLPIRBase(encode: int = 1, lib_path: Optional[int] = None, data_path: Optional[str] = None, license_code: str = '')[source]¶

Bases: abc.ABC

抽象类,作为各种NLPIR组件的基类,提供加载DLL等功能,大部分代码借鉴于pynlpir项目 Provides a low-level Python interface for NLPIR. Most of code in this model is copy/inspired from pynlpir

继承此类必须实现虚方法,实现对应不同组件的初始化和销毁动作. 为了使得类可以加载对应DLL,需要制定DLL名称,名称符合一般的操作系统对于动态链接库的命名规则:

linux: lib{Dll_name}32.so lib{Dll_name}64.so
macOS: lib{Dll_name}darwin.so 此处macOS与linux动态库命名方式一致,为了区分故加入darwin
windows: {Dll_name}32.dll, {Dll_name}64.dll

Parameters:

encode (int) – An encoding code provide from NLPIR’s header , defined in this package
lib_path (str) – The location of custom dynamic link library, None if use build-in lib
data_path (str) – The location of custom Data directory, None if use build-in Data directory. Can bu used in custom dictionary but dont want to change the build-in Data directory
license_code (str) – for license

Raises:

NLPIRException – Init the dynamic link library fail, can get an error message from dll

logger = <Logger nlpir.naive (WARNING)>¶: A logger using for all native nlpir functions

encode_map = {-1: 'utf-8', 0: 'gbk', 1: 'utf-8', 2: 'big5', 3: 'gbk', 4: 'utf-8'}¶

load_mode = None¶: use it if want load DLL in other mode

RTLD_LAZY = 1¶: lazy load DLL ,not supported for window, will be None on OS: windows

static byte_str_transform(func: __T__) → __T__[source]¶

一个包装器,作为装饰器使用,会自动将使用装饰器的函数的参数中的str转换为bytes, 在返回值中将bytes转换为str

此包装器只能在这个类的子类函数成员中使用,目的是简化动态链接库调用的编码转换问题

A wraps that automatically detect str parameter , transform to bytes and transform return value from bytes to str if it’s bytes.

This function is used for call the function from dynamic lib. This function can only use in NLPIRBase’s sub class

Parameters:	func – function

dll_name¶

Returns:	The name of dynamic link library, more info in class description

init_lib(data_path: str, encode: int, license_code: str) → int[source]¶

所有子类都需要实现此方法用于类初始化实例时调用, 由于各个库对应初始化不同,故改变此函数名称

Parameters:	data_path (str) – the location of Data , Data文件夹所在位置 encode – encode code define in NLPIR license_code (str) – license code for unlimited usage. common user ignore it
Returns:	1 success 0 fail

exit_lib() → bool[source]¶: 所有子类都需要实现此方法用于类析构(销毁)实例时调用, 由于各个库对应初始化不同,故改变此函数名称

get_dll_path(uname: platform.uname_result, lib_dir: str, is_64bit: bool) → str[source]¶

Parameters:	uname (platform.uname_result) – The platform identifier for the user’s system. lib_dir (str) – path to lib is_64bit (bool) – is 64bit or not
Returns:	the abspath of dll
Return type:	str

load_library(uname: platform.uname_result, is_64bit: Optional[bool] = None, lib_dir: Optional[str] = None) → Tuple[ctypes.CDLL, str][source]¶

Loads the NLPIR library appropriate for the user’s system. This function is called automatically when create a instance.

Parameters:	uname (platform.uname_result) – The platform identifier for the user’s system. is_64bit (bool) – Whether or not the user’s system is 64-bit. lib_dir (str) – The directory that contains the library files (defaults to `LIB_DIR`).
Returns:	a dynamic lib object
Return type:	tuple(ctypes.CDLL, str)
Raises:	RuntimeError – The user’s platform is not supported by NLPIR.

get_func(name: str, argtypes: Optional[list] = None, restype: Any = <class 'ctypes.c_int'>) → Callable[source]¶

Retrieves the corresponding NLPIR function.

Parameters:	name (str) – The name of the NLPIR function to get. argtypes (list) – A list of `ctypes` data types that correspond to the function’s argument types. restype (ctypes) – A `ctypes` data type that corresponds to the function’s return type (only needed if the return type isn’t `ctypes.c_int`).
Returns:	The exported function. It can be called like any other Python callable.
Return type:	Callable Function

get_last_error_msg() → str[source]¶: 对应每个组件获取异常信息的函数