Skip to content

gRPC Data Connector API

gRPC Data Connector API for version 12.207.0.

Package: qlik.connect

Data Connector Protobuf API specification

Methods

GetData

The standard way to send data. First send GetDataResponse as initial meta data. Then send DataChunk stream.

Parameters:

Type Description
DataRequest No description.

Response:

Type Description
DataChunk A structure for streaming field values. This message contains two parts. A value bucket part and a code part. Every transferred value have a string code and one or two numeric codes. These codes can be negative to indicate special things. -1 indicates null for example. Otherwise they can index into the value buckets. A value with a null string and a null number is a null value in the Qlik Engine. Number code have a special mechanism to escape an integer value inline with -2. Each chunk starts anywhere in any row and continues for an arbitary length and may wrap to another row. A DataChunk will typically transfer many rows. Each DataChunk must be equal or less in size to the default Grpc message size limit. This is currently 4 MB. 64 KB or slightly less is the optimal size for bandwith performance. But anything from 20 to 120 KB will give good performance if used with the hint option when writing. Very large strings can be sent in many DataChunk messages by using the -3 escape sequence.

GetMetaInfo

No description.

Parameters:

Type Description
MetaInfoRequest Empty

Response:

Type Description
MetaInfo Useful for debugging.

Definitions

ConnectionInfo

No description.

Fields:

Name Description Type Label Default
connectionString No description. string optional No default value.
user No description. string optional No default value.
password No description. string optional No default value.

DataChunk

A structure for streaming field values. This message contains two parts. A value bucket part and a code part. Every transferred value have a string code and one or two numeric codes. These codes can be negative to indicate special things. -1 indicates null for example. Otherwise they can index into the value buckets. A value with a null string and a null number is a null value in the Qlik Engine. Number code have a special mechanism to escape an integer value inline with -2. Each chunk starts anywhere in any row and continues for an arbitary length and may wrap to another row. A DataChunk will typically transfer many rows. Each DataChunk must be equal or less in size to the default Grpc message size limit. This is currently 4 MB. 64 KB or slightly less is the optimal size for bandwith performance. But anything from 20 to 120 KB will give good performance if used with the hint option when writing. Very large strings can be sent in many DataChunk messages by using the -3 escape sequence.

Fields:

Name Description Type Label Default
stringBucket Value buckets Strings the chunk string repeated No default value.
doubleBucket Doubles in the chunk. double repeated No default value.
stringCodes Code arrays 0 or greater is an index into stringBucket. -1 for a missning string. (null string) (no string value entry required). -2 for an empty string. (no string value entry required). A connector dont have to use the -2 optimization. Its perfectly fine to send an empty string as all other strings. sint32 repeated No default value.
numberCodes 0 or greater is an index into doubleBucket. -1 for a null number. -2 followed by an integer value. In this case there is no index into doubleValues. Instead -2 is followed by an actual field value number. This is a variable length encoded integer and will take up less space than a full double in most cases. A connector dont have to use the -2 optimization. Its perfectly fine to transfer all numbers in the double bucket. -3 is partial message special case. The string code of this value will be 0 or greater. That string will be appended to the previous value. One string value can be transferred in many chunks this way. The number value is always the transferred atomicly the first time. A connector dont have to use the -3 special case. A much simpler solution (if its acceptable) is to cap all strings to 3 MB or less so that they can be sent within the 4 MB limit. sint64 repeated No default value.

DataInfo

No description.

Fields:

Name Description Type Label Default
statement No description. string optional No default value.
parameters No description. Parameter repeated No default value.
firstRowsOnly Set by Engine to inform the Connector it will only use X rows. The Connector can then limit the number of rows fetched from the database. This is used with the FIRST prefix, or while debugging. If not set, the Engine will require all rows. OptionalInteger optional No default value.

DataRequest

No description.

Fields:

Name Description Type Label Default
connection No description. ConnectionInfo optional No default value.
sessionInfo No description. SessionInfo optional No default value.
parameters No description. DataInfo optional No default value.

FieldAttributes

Subset of classic Qlik Engine number format.

Fields:

Name Description Type Label Default
Type No description. FieldAttrType optional No default value.

FieldInfo

No description.

Fields:

Name Description Type Label Default
name No description. string optional No default value.
semanticType No description. SemanticType optional No default value.
fieldAttributes No description. FieldAttributes optional No default value.
tags Optional field description. Examples: key, text, ASCII. string repeated No default value.

GetDataResponse

Transferred as initial metadata with the name "x-qlik-getdata-bin".

Fields:

Name Description Type Label Default
fieldInfo No description. FieldInfo repeated No default value.
tableName No description. string optional No default value.

MetaInfo

Useful for debugging.

Fields:

Name Description Type Label Default
name Example: "AbcDb connector" string optional No default value.
version Version number in any format. Example: "1.0.0.0" or "1.23 Beta" or "Buildnumber: 1234" or "2018-Feb-06" string optional No default value.
developer Example: "Qlik" string optional No default value.

MetaInfoRequest

Empty

OptionalInteger

No description.

Fields:

Name Description Type Label Default
hasValue Used or not bool optional No default value.
value Integer value sint64 optional No default value.

Parameter

No description.

Fields:

Name Description Type Label Default
name No description. string optional No default value.
value No description. string optional No default value.

SessionInfo

No description.

Fields:

Name Description Type Label Default
user No description. string optional No default value.
sessionId No description. string optional No default value.
docId No description. string optional No default value.

Enums

FieldAttrType

Direct copy of FieldAttrType in Qlik Engine. How to display the data.

Name Number Description
UNKNOWN 0 No description.
TEXT 1 No description.
REAL 2 No description.
DATE 3 No description.
TIME 4 No description.
TIMESTAMP 5 No description.
INTERVAL 6 No description.
INTEGER 10 No description.
FIX 11 No description.
MONEY 12 No description.

SemanticType

How to interpret the data. If dates or times are already in the Qlik Engine format (fractional day since 1899-12-30), then the fastest way to import is to use SemanticType=DEFAULT. And transfer the date as a double and set FieldAttributes::Type = DATE. If the dates are in "days since 1904-01-01" format the best way is to add 1462 to them and send as 1899-12-30 dates.

Name Number Description
DEFAULT 0 The normal value.
UNIX_SECONDS_SINCE_1970_UTC 1 Transferred as double or integer.
ISO_8601 2 Date and time format. Transferred as string.

Scalar Value Types

double

cpp cs go java php python ruby
double double float64 double float float Float

float

cpp cs go java php python ruby
float float float32 float float float Float

int32

cpp cs go java php python ruby
int32 int int32 int integer int Bignum or Fixnum (as required)

int64

cpp cs go java php python ruby
int64 long int64 long integer/string int/long Bignum

uint32

cpp cs go java php python ruby
uint32 uint uint32 int integer int/long Bignum or Fixnum (as required)

uint64

cpp cs go java php python ruby
uint64 ulong uint64 long integer/string int/long Bignum or Fixnum (as required)

sint32

cpp cs go java php python ruby
int32 int int32 int integer int Bignum or Fixnum (as required)

sint64

cpp cs go java php python ruby
int64 long int64 long integer/string int/long Bignum

fixed32

cpp cs go java php python ruby
uint32 uint uint32 int integer int Bignum or Fixnum (as required)

fixed64

cpp cs go java php python ruby
uint64 ulong uint64 long integer/string int/long Bignum

sfixed32

cpp cs go java php python ruby
int32 int int32 int integer int Bignum or Fixnum (as required)

sfixed64

cpp cs go java php python ruby
int64 long int64 long integer/string int/long Bignum

bool

cpp cs go java php python ruby
bool bool bool boolean boolean boolean TrueClass/FalseClass

string

cpp cs go java php python ruby
string string string String string str/unicode String (UTF-8)

bytes

cpp cs go java php python ruby
string ByteString []byte ByteString string str String (ASCII-8BIT)