Talend 从Excel导入Saleforce数据(一) 直接从salesforce lookup 性能的噩梦

 

 

速度的瓶颈是在查询Sales force是否有该电话号码的联系人资料.

TMap属性的 lookup Model, 如果用Load Once, 则会把SaleForce的contact全部load下来,数据量很大,这是不能接受.

只能改成Reload at Each Row, 然后定义一个globalMap key,

 

然后在TsalesForceInput的SOQL的查询语句改成

 

"select Id,IsDeleted,MasterRecordId,AccountId,IsPersonAccount,LastName,FirstName,Salutation,Name,RecordTypeId,

OtherStreet,OtherCity,OtherState,OtherPostalCode,OtherCountry,OtherLatitude,OtherLongitude,OtherAddress,

MailingStreet,MailingCity,MailingState,MailingPostalCode,MailingCountry,MailingLatitude,MailingLongitude,

MailingAddress,Phone,Fax,MobilePhone,HomePhone,OtherPhone,AssistantPhone,ReportsToId,Email,Title,Department,

AssistantName,LeadSource,Birthdate,Description,CurrencyIsoCode,OwnerId,HasOptedOutOfEmail,HasOptedOutOfFax,

DoNotCall,CreatedDate,CreatedById,LastModifiedDate,LastModifiedById,SystemModstamp,LastActivityDate,

LastCURequestDate,LastCUUpdateDate,LastViewedDate,LastReferencedDate,EmailBouncedReason,EmailBouncedDate,

IsEmailBounced,PhotoUrl,Jigsaw,JigsawContactId,Addway__c,AlertSet_c__c,City__c,CustomerL__c,MemberLevel__c,

States__c,Address2__c,Address3__c,Address__c,Age__c,ComOther__c,ComQQ__c,Comwechat__c,Comweibo__c,Division__c,

JoinOrNot__c,Joindate__c,MainName2ID__c,MainName2__c,MainNameID__c,MemberInteger__c,MemberNumber__c,Memo2__c,

Memo__c,ExternalID__c,NextCallBack__c,OtherID__c,PostDate__c,Tel2__c,Tel3__c,testClick__c,testautonumber__c,

MemberActivties__c,EmailAddress__c,Subsidiaries__c,IsDateLoader__c,Line__c,English_Name__c,Want_Type__c,

Get_Info__c,Total_Point__c,Exchanged_Point__c,Remaining_Point__c,website__c,TWMemberLevel__c,sex__c,Identity_CardID__c,

newsletter__c,Ostricare_baby__c,babecare__c,karihome__c,Ostricare_Level__c,Babecare_Level__c,Karihome_Level__c,

Ostricare_adult__c,Ostricare_adult_Level__c,A_DERMA__c,DUCRAY__c,NOV__c,BIO_OIL__c,AgeNum__c,Others__c,Quicure__c

from Contact where phone='"+ globalMap.get("client.Phone")+"'"

 

这样就把导入Excel的每一行数据的电话去SalesForce查询资料. 每次查询约需2s. 

 

在数据量大时,这个性能也是不能接受的. 还是建议先把资料导出csv, lookup-modual 改成Load once, 而不是Reload at each row