Solr DataImportHandler with SQL Server

10,777

It seems that the datasource is not recognized in the entity declaration because the proper attribute to use is dataSource, not datasource

Share:
10,777
Admin
Author by

Admin

Updated on October 12, 2022

Comments

  • Admin
    Admin over 1 year

    I'm having a problem getting Solr to talk to Microsoft SQL Server via the Microsoft JDBC Driver. I have the handler registered in solrconfig.xml:

    <requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
      <lst name="defaults">
        <str name="config">C:\Program Files\Apache Software Foundation\Tomcat 6.0\Solr\conf\data-config.xml</str>
      </lst>
    </requestHandler>
    

    In data-config.xml I have a data source and a document defined:

    <?xml version="1.0" encoding="UTF-8" ?>
    <dataConfig>
      <dataSource type="JdbcDataSource" name="ds1"
        driver="com.microsoft.sqlserver.jdbc.SQLServerDriver" 
        url="jdbc:sqlserver://localhost;databaseName=myDB;responseBuffering=adaptive;"
        user="xxxx"
        password="xxxx"
        readOnly="true"
      />
    
      <document name="members">
        <entity name="member" datasource="ds1" pk="id"
          query = "select 
            MemberID as id,
            UserName as userName,
            FirstName as firstName,
            LastName as lastName,
            Birthday as birthday,
            PrimaryEmail as primaryEmail,
            PersonalStatement as personalStatement
            from member"
          transformer="DateFormatTransformer">
          <field column="Birthday" name="birthday" dateTimeFormat="yyyy-MM-dd" />
        </entity>      
      </document>
    </dataConfig>
    

    The columns are fairly irrelevant - I just wanted to start with a few items, including a date column. The Solr schema.xml has some fields defined:

    <field name="id" type="tlong" indexed="true" stored="true" required="true" /> 
    <field name="userName" type="text" indexed="true" stored="true" />
    <field name="firstName" type="text" indexed="true" stored="true" />
    <field name="lastName" type="text" indexed="true" stored="true" />
    <field name="birthday" type="tdate" indexed="true" stored="true" />
    <field name="primaryEmail" type="text" indexed="true" stored="true" />
    <field name="personalStatement" type="text" indexed="true" stored="true" />
    

    When I attempt an import, the log shows an exception building the datasource:

    Jun 26, 2010 10:24:48 PM org.apache.solr.handler.dataimport.DataImporter doFullImport INFO: Starting Full Import Jun 26, 2010 10:24:48 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={clean=false&commit=true&command=full-import&qt=/dataimport} status=0 QTime=7 Jun 26, 2010 10:24:48 PM org.apache.solr.handler.dataimport.SolrWriter readIndexerProperties WARNING: Unable to read: dataimport.properties Jun 26, 2010 10:24:48 PM org.apache.solr.handler.dataimport.DataImporter doFullImport SEVERE: Full Import failed org.apache.solr.handler.dataimport.DataImportHandlerException: No dataSource :null available for entity :member Processing Document # 1 at org.apache.solr.handler.dataimport.DataImporter.getDataSourceInstance(DataImporter.java:279) at org.apache.solr.handler.dataimport.ContextImpl.getDataSource(ContextImpl.java:93) at org.apache.solr.handler.dataimport.SqlEntityProcessor.init(SqlEntityProcessor.java:52) at org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:71) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:319) at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:242) at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:180) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:331) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:389) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:370) Jun 26, 2010 10:24:48 PM org.apache.solr.update.DirectUpdateHandler2 rollback INFO: start rollback Jun 26, 2010 10:24:48 PM org.apache.solr.update.DirectUpdateHandler2 rollback INFO: end_rollback Jun 26, 2010 10:24:54 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={clean=false&commit=true&command=status&qt=/dataimport} status=0 QTime=0

    I've read the FAQ and documentation, looked at as many sources as I can find, and I just can't get past this error. What am I doing wrong? The error "Unable to read: dataimport.properties" appears to be shown any time there is any issue with the configuration. I can't find my mistake.