ZHANG Yinsheng（張寅升），LI Haomin（李昊旻），ZHENG Xiang（鄭 翔），GE Caixia（葛彩霞），HUANG Zhenzhen（黃震震），JIA Zheng（賈 崢），DUAN Huilong（段會(huì)龍）

1College of Biomedical Engineering and Instrument Science，Zhejiang University，Hangzhou 310027，China

2School of Computer Science ＆Information Engineering，Zhejiang Gongshang University，Hangzhou 310018，China

3The Children's Hosptial of Zhejiang University，Hangzhou 310003，China

4The Institute of Translational Medicine，Zhejiang University，Hangzhou 310027，China

Introduction

Globally，healthcare organizations fail to use evidence optimally.A large gulf remains between what we know and what we practice.In 1973， Wennberg and Gittelsohn published their landmark article demonstrating substantial variation among different healthcare service providers［1］.Forty years later，this had not been changed.One study in 2003 showed that only 54.9% US patients received recommended care［2］.Similar findings have been reported worldwide in both developed and developing settings［3］.To address this problem，knowledge translation has been proposed and widely studied over the past decades.According to the Canadian Institutes of Health Research（CIHR），knowledge translation has been defined as “a dynamic and iterative process that includes the synthesis，dissemination，exchange and ethically sound application of knowledge to provide more effective health services and products and strengthen the healthcare system ”［4］.Knowledge translation has now become the biggest challenge faced by evidence based medicine［5］.Knowledge translation includes not only dissemination of knowledge，but also“ethically sound application of knowledge”in healthcare system［6］.Clinical decision support（CDS）systems，which translated knowledge into computable logic，have been proved to be such an effective knowledge translation tool in improving healthcare quality and reducing cost［7-8］.This paper aims to design a technical framework to facilitate the development and usage of CDS applications.

Related works in knowledge translation field include healthcare domain knowledge modeling［9-12］， CDS intervention modeling［13］，knowledge management［14-15］，knowledge dissemination［16］， context-aware knowledge retrieval［17］，etc.There are also researches on motivation frameworks or architectures to facilitate knowledge translation using computerized CDS as knowledge tools，such as knowledge management repository II（KMR-II）［18］and evidence-based medicine eletronic decision support（EBMeDS）［19］.KMR-II is a nationwide CDS infrastructure supported by US federal agencies.It includes fundamental service components including event driven mechanism，rule engine and predictive modeling.KMR-II focuses on using open standards and technologies，such as HL7and service oriented architecture（SOA），to achieve distributed decision support services.EBMeDS is a project supported by Belgium government to promote evidence based medicine.EBMeDS is backed by a nationwide knowledge base which uses server side JavaScript for encoding guideline rules.Its front end uses web service to interact with electronic medical record（EMR）.These studies mainly concern distributed and interoperable CDS services.Our framework focuses on defining and providing a set of fundamental services for building knowledge-driven CDS applications， such as knowledge query，inference engine，data acquisition，and normalized CDS intervention.Various CDS applications can be registered to specific clinical events and executed in a context-aware way.Based on such an infrastructure，CDS applications can be more easily developed and integrated into clinical workflows.The framework was implemented in a 2600-bed Chinese hospital（Dayi Hospital，Shanxi province，China）.Initial evaluation shows the framework is promising in reducing the time and cost for developing CDS applications.

1 Method

The knowledge translation framework comprises of four components：a unified medical knowledge ontology，a comprehensive knowledge base，a knowledge translation platform，and CDS applications built upon the platform.The unified medical ontology incorporates various types of knowledge，such as diagnostic knowledge， treatment knowledge，and pharmaceutical knowledge.Based on the ontology，a comprehensive knowledge base is constructed.Clinical experts translate the latest knowledge into computable forms and periodically update the content of knowledge base.Upon the knowledge base，a knowledge translation platform is built to provide fundamental services.Based on the knowledge translation platform，various CDS applications can be easily developed and integrated in clinical workflow.

1.1 Unified medical knowledge ontology

Existing CDS systems are dedicated for specific purposes，and use heterogeneous knowledge bases.There lacks general-purpose knowledge bases that are able to cover wide range of medical knowledge and support diverse clinical needs.Therefore，we constructed a unified ontology to manage several typical types of medical knowledge，including diagnostic knowledge，treatment knowledge， pharmaceutical knowledge，as well as knowledge reference（Fig.1）.

Fig.1 A unified ontology that contains various medical knowledge

Diagnostic knowledge includes clinical problems and corresponding diagnostic rules.Diagnostic rules are defined with a batch of context items（e.g.blood sugar）which are classified into several semantic categories（lab test result，vital sign，etc.）.These rules are evaluated by an inference engine based on related facts（e.g.blood sugar is 10mmol／L）.The diagnostic knowledge can be used to support diagnostic CDS applications.

Treatment knowledge includes recommended care protocols for clinical problems，including order sets and clinical pathways.In this ontology，order set and clinical pathway are both represented in multiple phases.Each phase contains a set of predefined orders such as medication，diet，or lab test request.Care protocols are effective tools to translate knowledge into clinical practice.They are represented as normalized CDS interventions that can be directly translated into medical orders in physician workstation.

Pharmaceutical knowledge includes drug use rules for validating medication order and alerting physicians，which can be used to support medication-relation CDS applications.

Knowledge reference contains various evidence based medicine （EBM）documents，such as clinical practice guideline，meta-analysis，system review，and randomized clinical trial.These literatures can be tagged to other knowledge artifacts，such as diagnostic rule and care protocol.Knowledge reference can be displayed to end user through context-aware information retrieval CDS applications，such as infobutton［20］.

The ontology also contains related concepts for natural language processing （NLP）.An NLP task uses specific algorithm and lexicon to extract data from free text clinical documents.NLP is a complementary technology for traditional structured data acquisition，such as retrieving data directly from database.Extracted data are further used as facts for inference engine or to drive various CDS applications.

1.2 Comprehensive knowledge base

The unified medical knowledge ontology gives a formal conceptualization for medical knowledge.It defines what kinds of knowledge services can be provided.The ontology needs to be technically consolidated as a knowledge base in the information system.We use dentity framework［21］to construct knowledge base.Entity framework is Microsoft's implementation of object relation mapping （ORM ）technology［22］.The ontology was built by entity framework's conceptual model editor，and then translated into SQL Server database schema and corresponding programming language objects.The content of knowledge base can be updated and maintained by clinical experts through a knowledge authoring web portal，which will be described later.

In clinical settings，knowledge management has become a common challenge faced by all decision makers［23］.However，most physicians do not have the capability or time to appraise， understand and apply relevant medical knowledge.The knowledge management by expert panel in a central knowledge base is a feasible solution to this challenge.

1.3 Knowledge translation platform

For effective knowledge translation，the information stored in knowledge base must be translated into effective CDS applications［24］.To drive this process，a knowledge translation platform is designed as Fig.2，where EMR is electronic medical record，CDR is clinical data repository，and NLP is natural language processing.The platform is a bridge between the knowledge base and CDS applications.The platform provides four fundamental services used for building various CDS applications.

（1）Inference engine

An inference engine is an indispensable component for many CDS applications［25-28］.Famous inference engines include C language integrated production system（CLIPS）［29］and Drools［30］.In this platform，we developed our own rulebased inference engine based on another tool for language recognition（ANTLR）［31］project.To further complement the rule based reasoning，case based reasoning［32］will be considered in future.

（2）Data acquisition

Fig.2 Knowledge translation platform

Data acquisition is a prerequisite procedure for CDS applications.Besides traditional structured data acquisition，the platform provides NLP for extracting data from free text documents，where lots of valuable clinical data reside.In this platform，user can configure NLP tasks with chosen algorithms and lexicons to process specific document types.This can improve the efficiency and accuracy of NLP extraction.Extracted data are used as facts for inference engine or used to drive various CDS applications.It is also a trend to extend images for data retrieval and analysis［33］，which will be considered in future.

（3）Normalized CDS intervention

It is important to represent CDS interventions in both human-readable and machine-interpretable format，so that they can be readily understood and applied［34］.The unified ontology has defined the structure and format of care protocols in the knowledge base，so the platform is able to provide such normalized interventions defined in the knowledge base.

（4）Knowledge query

The knowledge base allows user to upload and manage various EBM literatures，such as clinical practice guideline，meta-analysis，system review，randomized clinical trial，and drug fact sheet.The platform provides a knowledge query service that retrieves literatures by specifying query criteria.

The four services form an infrastructure for developing knowledge-driven CDS applications.A software development kit（SDK）that interoperates with these service application programming interfaces（APIs）is provided to facilitate engineers to develop CDS applications.Engineers may also not use the SDK but develop independent CDS applications.

After CDS applications are developed，they can be registered to specific clinical events in clinical workflows.In runtime，these applications are automatically launched at the right time.A typical CDS application execution process is shown in Fig.2.

（1）A clinical event source sends event to trigger the reasoning process of the inference engine.An open event interface designed in the framework could accept predefined clinical events from different event sources to trigger the inference engine.In clinical settings，the integration engine［35］which works as a portal or gateway between various systems is the ideal event source for this framework.Typical clinical events include patient admission，or arrival of lab test result.Each CDS application can be registered to specific clinical event.

（2）The data acquisition service retrieves facts either from CDR or from free-text-style clinical documents through NLP.The retrieved data are used by CDS application or fed into inference engine.

（3）Based on the context information，the right CDS application is launched.For example，if the current clinical event is an arrival of new lab test result，the inference process can generate clinical problems using the diagnostic rules and retrieved data from step（2）.The inference engine can also validate medication orders by drug use rules.Based on clinical event type and inference result，registered CDS applications will be pushed to end users.It could be a CDS application that simply displays the detected clinical problems or drug use alerts to users.It could also be a CDS application that recommends normalized CDS interventions，i.e.order sets or clinical pathway，for detected problems.

2 Results

The knowledge translation framework was implemented in a 2600-bed Chinese hospital in September 2013.A knowledge authoring web portal has been provided to edit the knowledge base.With the knowledge base，a knowledge translation platform was developed and several CDS applications were constructed upon the platform.

2.1 Knowledge authoring web portal and knowledge base

The knowledge authoring web portal has been used by clinicalexperts to build up localized knowledge base.The web portal is designed as a multi-language web application（demonstration can be accessed at http：／／www.cktp.org：8006）.Figure 3（a）shows the clinical problem edit page.Diagnostic rule and care protocol edit pages are shown in Figs.3（b）and（c）.

Through this knowledge authoring web portal，the knowledge base now contains 27 960clinical problems from international classification of diseases tenth revision（ICD10）［36］，463care protocols created according to China Ministry of Health（MOH）published clinical pathways［37］and other public resources，4 138EBM literatures，and 1 986 drugs from the hospital's pharmacy system.Drug alert rules have been defined for most frequently used medicine.The rules includes dosage，administration route，frequency，skin test，solvent，pregnancy risk，drug-drug interaction（DDI），and contra-indications.

Fig.3 Knowledge authoring web portal：（a）clinical problem edit page；（b）diagnostic rule edit page；（c）care protocol edit page

2.2 Knowledge translation platform and CDS applications

The knowledge translation platform was developed to be an infrastructure for constructing various CDS applications.The infrastructure provides following services.（1）An inference engine based on ANTLR［31］，as well as a corresponding rule edit page （Fig.3（b））.（2）A data acquisition service that can retrive structured data directly from CDR，or from free-text clinical documents by NLP（Fig.4）.Lots of data integration efforts were made between knowledge translation platform and CDR to provide basic data acquisition.The platform also provides NLP for nonstructured data acquisition.User can configure NLP tasks with different algorithms and lexicons，for specific document types，as shown in Fig.4.（3）Normalized CDS interventions，that are defined as order sets or clinical pathway in the knowledge base（Fig.3（c））.（4）Knowledge query that is enabled by conditional querying EBM literatures in knowledge base.

g.4 Configuring NLP task by choosing algorithm and lexicon for specific document type

Based on the knowledge translation platform，several CDS applications have been developed and deployed in hospital.Figure 5shows the CDS applications implemented in computerized physician order entry（CPOE）.Figure 5（a）is the list of clinical problems detected by the diagnostic CDS.Related diagnostic rule and facts for each problem is traceable.Figure 5（b）shows recommended care protocols for clinical problem.These protocols are normalized interventions and can be directly added as medical orders into the CPOE workbench when applied.Figure 5（c）is a notification area which displays alerts generated by drug alert CDS.The drug alert CDS is fired each time user submits orders.The notification is designed as non-blocking message.After clicking the alert message，CPOE will navigate to the corresponding medication order.Figure 5（d）is an infobutton that retrieves drug fact sheet.Figure 5（e）shows the retrieved literature by knowledge query service.

Besides the above CDS applications that are integrated into CPOE，a web-based fetus growth evaluation tool（Fig.6）was also developed.This application is enabled by NLP and driven by extracted data from pregnancy ultrasound report.

Fig.5 CPOE in physician's workstation：（a）clinical problem list；（b）recommended care protocols；（c）notification area for drug alert；（d）infobutton；（e）retrieved knowledge display

Fig.6 A free text driven CDS application that evaluates fetus growth

By the knowledge translation platform，legacy and external CDS applications can also be integrated.They can be automatically launched under certain context and executed as independent CDS applications.For example，when physician edits medical order of insulin，the eventdriven mechanism of platform will push insulin dosage advisor application to physician.

3 Discussion

To assess how much the framework improved the development efficiency of CDS applications，we interviewed the development team.The project manager said the software development effort reduction was obvious.Reduced effort was achieved by the fundamental services provided by the platform.For example，when developing diagnostic and drug alert CDS，if without these services，additional efforts are needed for developing inference engine and retrieving data from external systems.This has shown the framework is promising in improving the development efficiency of CDS applications.However，this is only one aspect of the knowledge translation framework，a more comprehensive evaluation is needed to assess how the framework influences the overall knowledge dissemination and translation process.

A systematic evaluation methodology is the biggest challenge for future research.We have been considering several approaches.The first is observational method［38］.A knowledge translation is deemed effective when practice is altered.For example，after a new clinical problem is detected or a drug alert is fired， physician takes corresponding actions，such as creating new medical orders or revising existing medical orders.If such behavioral changes are observed，knowledge translation is regarded as successful.The second approach is key performance indicator（KPI）comparison.A list of KPI is identified and measured before and after the framework is deployed.Significant difference indicates the framework has an influence on clinical practice.The most difficult part of this approach is the identification and selection of KPIs.General performance indicators include length of stay， cost，comorbidity rate，medication error rate，etc.There are also clinical indicators related to specific specialties and diseases.KPIs should be chosen based on designed clinical test settings.The third approach is questionnaire.To design a high quality set of questions is the most difficult work.Appraisal tools such as QAS-99［39］can be used.

Another important future work is knowledge content development.A knowledge base with plenty of contents can support more diverse and effective knowledge services.Knowledge management is a laborious and time-consuming job，and can take 50%of total effort in developing a CDS application［40］.An expert committee of clinicians，pharmacists，nurses，and medical insurance staff，should be established to periodically create， revise and curate knowledge content.In future，we will consider exporting the knowledge content in standard formats for better knowledge sharing between healthcare institutes.

4 Conclusions

Translating medical knowledge into clinical practice has always been an urgent need for healthcare organizations.A knowledge translation framework is proposed in this paper，and has been implemented in a Chinese hospital.We hope this pioneer effort can eventually promote the knowledge translation process in more healthcare institutes in China.

［1］Wennberg J，Gittelsohn A.Small Area Variations in Health Care Delivery：a Population-Based Health Information System can Guide Planning and Regulatory Decision-Making ［J］.Science，1973，182（4117）：1102-1108.

［2］McGlynn E A，Asch S M，Adams J，et al.The Quality of Health Care Delivered to Adults in the United States［J］.New England Journal of Medicine，2003，348（26）：2635-2645.

［3］Grol R.Successes and Failures in the Implementation of Evidence-Based Guidelines for Clinical Practice［J］.Medical Care，2001，39（8）：II-46-II-54.

［4］Tetroe J.Knowledge Translation at the Canadian Institutes of Health Research：a Primer［J］.Focus Technical Brief，2007（18）：1-8.

［5］Guyatt G，Cook D，Haynes B.Evidence Based Medicine has Come a Long Way［J］.British Medical Journal：International Edition，2004，329（7473）：990.

［6］Canadian Institutes of Health Research.About Knowledge Translation ＆ Commercialization ［EB／OL］.（2014-02-27）［2014］.http：／／www.cihr-irsc.gc.ca／e／29418.html.

［7］Sim I，Gorman P，Greenes R A，et al.Clinical Decision Support Systems for the Practice of Evidence-Based Medicine［J］.Journal of the American Medical Informatics Association，2001，8（6）：527-534.

［8］Kawamoto K，Houlihan C A，Balas E A，et al.Improving Clinical Practice Using Clinical Decision Support Systems：a Systematic Review of Trials to Identify Features Critical to Success［J］.BMJ，2005，330（7494）：765.

［9］Adlassnig K P.Arden Syntax：Wissens Repr?sentation Undverarbeitung in der Medizin ［EB／OL］.（2011）［2014］.http：／／www.hl7.de／download／veranstaltungen／jahrestagungen／2011／Adlassnig.pdf.

［10］Wang D，Peleg M，Tu S W，et al.Design and Implementation of the GLIF3 Guideline Execution Engine ［J］.Journal of Biomedical Informatics，2004，37（5）：305-318.

［11］de Clercq P，Blom J，Korsten H，et al.Approaches for Creating Computer-Interpretable Guidelines that Facilitate Decision Support ［J］.Artificial Intelligence in Medicine，2004，31（1）：1-27.

［12］Tu S W，Campbell J R，Glasgow J，et al.The SAGE Guideline Model：Achievements and Overview［J］.Journal of the American Medical Informatics Association，2007，14（5）：589-598.

［13］Chapman S.Seeking Clinical Decision-Support Standards［J］.For the Record，2012，24（18）：14.

［14］Popov B，Kiryakov A，Kirilov A，et al.KIM-Semantic Annotation Platform ［C］.The Semantic Web-ISWC 2003，Sanibel Island，F(xiàn)lorida，USA，2003：834-849.

［15］Abidi S S R.Knowledge Management in Healthcare：towards“Knowledge-Driven” Decision-Support Services ［J］.International Journal of Medical Informatics，2001，63（1）：5-18.

［16］Boxwala A A，Rocha B H，Maviglia S，et al.A Multi-layered Framework for Disseminating Knowledge for Computer-Based Decision Support ［J］.Journal of the American Medical Informatics Association，2011，18（s1）：i132-i139.

［17］Maviglia S M，Yoon C S，Bates D W，et al.KnowledgeLink：Impact of Context-Sensitive Information Retrieval on Clinicians'Information Needs［J］.Journal of the American Medical Informatics Association，2006，13（1）：67-73.

［18］Sottara D，F(xiàn)ry E，Aliverti E，et al.Managing High Disease Risk Factors：a Use Case in the KMR-II Healthcare Infrastructure［C］.RuleML2011＠ BRF Challenge，Westin Diplomat，F(xiàn)t Lauderdale，F(xiàn)lorida，2011：79.

［19］Heselmans A，Aertgeerts B，Donceel P，et al.Family Physicians'Perceptions and Use of Electronic Clinical Decision Support during the First Year of Implementation［J］.Journal of Medical Systems，2012，36（6）：3677-3684.

［20］Del Fiol G，Huser V，Strasberg H R，et al.Implementations of the HL7 Context-Aware Knowledge Retrieval（“Infobutton ”） Standard： Challenges， Strengths，Limitations，and Uptake ［J］.Journal of Biomedical Informatics，2012，45（4）：726-735.

［21］Adya A，Blakeley J A，Melnik S，et al.Anatomy of the Ado.Net Entity Framework［C］.Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data，Beijing，China，2007：877-888.

［22］He Z，Chen Z G.Research and Application about Object／Relation Mapping Framework［J］.Computer Engineering and Applications，2003，39（26）：188-191.（in Chinese）

［23］Cabana M D，Rand C S，Powe N R，et al.Why don't Physicians Follow Clinical Practice Guidelines？［J］.JAMA：the Journal of the American Medical Association，1999，282（15）：1458-1465.

［24］Shiffman R N，Michel G，Essaihi A，et al.Bridging the Guideline Implementation Gap：a Systematic，Document-Centered Approach to Guideline Implementation［J］.Journal of the American Medical Informatics Association，2004，11（5）：418-426.

［25］Hussain S，Abidi S R，Abidi S S R.Semantic Web Framework for Knowledge-Centric Clinical Decision Support Systems［C］.AIME 2007，Amsterdam，The Netherlands，2007：451-455.

［26］Achour S L，Dojat M，Rieux C，et al.A UMLS-Based Knowledge Acquisition Tool for Rule-Based Clinical Decision Support System Development［J］.Journal of the American Medical Informatics Association，2001，8（4）：351-360.

［27］White S C.Decision-Support Systems in Dentistry ［J］.Journal of Dental Education，1996，60（1）：47-63.

［28］Kashyap V，Morales A，Hongsermeier T.On Implementing Clinical Decision Support： Achieving Scalability and Maintainability by Combining Business Rules and Ontologies［C］.AMIA Annual Symposium Proceedings，Washington DC，USA，2006：414-418.

［29］Wygant R M.CLIPS—a Powerful Development and Delivery Expert System Tool［J］.Computers ＆Industrial Engineering，1989，17（1）：546-549.

［30］Liu W.Introduction and Implementation of Drools—a Rule Engine Based Java［J］.Microcomputer Applications，2005，26（6）：717-721.（in Chinese）

［31］Parr T J，Quong R W.ANTLR：a Predicated‐ LL（k）Parser Generator ［J］.Software：Practice and Experience，1995，25（7）：789-810.

［32］Aamodt A，Plaza E.Case-Based Reasoning：Foundational Issues，Methodological Variations，and System Approaches［J］.AI Communications，1994，7（1）：39-59.

［33］Sen A，Banerjee A，Sinha A P，et al.Clinical Decision Support：Converging toward an Integrated Architecture［J］.Journal of Biomedical Informatics，2012，45（5）：1009-1017.

［34］Osheroff J A，Teich J M，Middleton B，et al.A Roadmap for National Action on Clinical Decision Support［J］.Journal of the American Medical Informatics Association，2007，14（2）：141-145.

［35］Zhao C H，Duan H L，Lu X D.An Integration Approach of Healthcare Information System ［C ］.International Conference on BioMedical Engineering and Informatics，Sanya，China，2008：606-609.

［36］World Health Organization.International Classification of Diseases（ICD）［EB／OL］.（2010）［2013］.http：／／www.who.int／classifications／icd／en／.

［37］China Ministry of Health.The Medical Administrative Department of the Ministry of Health Held the Clinical Pathway Management Work Conference in 2012［EB／OL］.（2012） ［2012］.http：／／www.gov.cn／gzdt／2012-05／14／content＿2136918.htm.

［38］Centers for Disease Control and Prevention.Data Collection Methods for Program Evaluation：Observation ［EB／OL］.（2008） ［2014］.http：／／www.cdc.gov／healthyyouth／evaluation／pdf／brief16.pdf.

［39］Willis G B，Lessler J T.Question Appraisal System QAS-99［J］.［EB／OL］.（1999）［2014］.http：／／applied research.cancer.gov／areas／cognitive／qas99.pdf.

［40］Field T S，Rochon P，Lee M，et al.Costs Associated with Developing and Implementing a Computerized Clinical Decision Support System for Medication Dosing for Patients with Renal Insufficiency in the Long-Term Care Setting［J］.Journal of the American Medical Informatics Association，2008，15（4）：466-472.