可可西

UE4之DDC缓存

什么是DDC(DerivedDataCache)?简单来说,是一些缓存文件。在使用Editor的过程中,有可能会在编辑某些文件,或者使用某些文件的时候产生额外的数据。

为了避免每次都需要重新产生一次数据,所以第一次产生完数据之后,会将数据序列化,并以缓存的形式保存下来。

DerivedDataCache目录包含了为引用的内容而生成的衍生数据文件。如果引用内容没有可用的缓存文件,则会极大增加载入时间。

 

DerivedDataCache代码

获取FDerivedDataCacheInterface全局单例对象

/*** UnrealEngine\Engine\Source\Runtime\Core\Public\Misc\CoreMisc.h ***/
/** Return the DDC interface, if it is available, otherwise return NULL **/
CORE_API class FDerivedDataCacheInterface* GetDerivedDataCache();

/** Return the DDC interface, fatal error if it is not available. **/
CORE_API class FDerivedDataCacheInterface& GetDerivedDataCacheRef();

// ---------------------------------------------------------------------------------------

/*** UnrealEngine\Engine\Source\Runtime\Core\Private\Misc\CoreMisc.cpp ***/
class FDerivedDataCacheInterface* GetDerivedDataCache()
{
    static class FDerivedDataCacheInterface* SingletonInterface = NULL;
    if (!FPlatformProperties::RequiresCookedData())
    {
        static bool bInitialized = false;
        if (!bInitialized)
        {
            check(IsInGameThread());
            bInitialized = true;
            class IDerivedDataCacheModule* Module = FModuleManager::LoadModulePtr<IDerivedDataCacheModule>("DerivedDataCache");
            if (Module)
            {
                SingletonInterface = &Module->GetDDC();
            }
        }
    }
    return SingletonInterface;
}

class FDerivedDataCacheInterface& GetDerivedDataCacheRef()
{
    class FDerivedDataCacheInterface* SingletonInterface = GetDerivedDataCache();
    if (!SingletonInterface)
    {
        UE_LOG(LogInit, Fatal, TEXT("Derived Data Cache was requested, but not available."));
        CA_ASSUME( SingletonInterface != NULL );    // Suppress static analysis warning in unreachable code (fatal error)
    }
    return *SingletonInterface;
}

 

DerivedDataCache插件

 

将数据写入DDC缓存

/*** UnrealEngine\Engine\Source\Runtime\Engine\Private\Texture2D.cpp ***/
#if
WITH_EDITORONLY_DATA uint32 FTexture2DMipMap::StoreInDerivedDataCache(const FString& InDerivedDataKey, const FStringView& TextureName) { int32 BulkDataSizeInBytes = BulkData.GetBulkDataSize(); check(BulkDataSizeInBytes > 0); TArray<uint8> DerivedData; FMemoryWriter Ar(DerivedData, /*bIsPersistent=*/ true); Ar << BulkDataSizeInBytes; { void* BulkMipData = BulkData.Lock(LOCK_READ_ONLY); Ar.Serialize(BulkMipData, BulkDataSizeInBytes); BulkData.Unlock(); } const uint32 Result = DerivedData.Num(); GetDerivedDataCacheRef().Put(*InDerivedDataKey, DerivedData, TextureName); DerivedDataKey = InDerivedDataKey; BulkData.RemoveBulkData(); return Result; } #endif // #if WITH_EDITORONLY_DATA

 

步骤如下:

① 创建一个TArray<uint8>类型的数组,用来存放要保存的数据
② 创建一个FMemoryWriter,构造的时候使用上面创建的数组
③ 往FMemoryWriter写入数据,可以使用<<符号压入数据,也可以使用Serialize()来压
④ 使用FDerivedDataCacheInterface::Put()将第一步创建的数组设置进DDC,注意注意DDC的Key需要遵循一定的规则,日后才能顺利地使用同一个Key重新取出

引擎中调用GetDerivedDataCacheRef().Put()函数的地方:

D:\svn\UnrealEngine\Engine\Plugins\Compositing\OpenColorIO\Source\OpenColorIO\Private\OpenColorIOShaderMap.cpp(347):	GetDerivedDataCacheRef().Put(*GetOpenColorIOShaderMapKeyString(GetContent()->ShaderMapId, GetShaderPlatform()), SaveData, FStringView(*GetFriendlyName()));
D:\svn\UnrealEngine\Engine\Plugins\Editor\AssetSearch\Source\Private\AssetSearchManager.cpp(685):					GetDerivedDataCacheRef().Put(*InDDCKey, IndexedJsonUTF8View, InAssetData.ObjectPath.ToString(), false);
D:\svn\UnrealEngine\Engine\Plugins\FX\Niagara\Source\NiagaraShader\Private\NiagaraShader.cpp(617):	GetDerivedDataCacheRef().Put(*GetNiagaraShaderMapKeyString(GetContent()->ShaderMapId, GetShaderPlatform()), SaveData, FStringView(*GetFriendlyName()));
D:\svn\UnrealEngine\Engine\Plugins\FX\Niagara\Source\Niagara\Private\NiagaraScript.cpp(1795):				GetDerivedDataCacheRef().Put(*GetNiagaraDDCKeyString(), OutData, GetPathName());
D:\svn\UnrealEngine\Engine\Plugins\FX\Niagara\Source\Niagara\Private\NiagaraSpriteRendererProperties.cpp(559):			GetDerivedDataCacheRef().Put(*KeyString, Data, GetPathName());
D:\svn\UnrealEngine\Engine\Plugins\FX\Niagara\Source\Niagara\Private\NiagaraSystem.cpp(1815):		GetDerivedDataCacheRef().Put(*DDCKey, OutData, GetPathName());
D:\svn\UnrealEngine\Engine\Plugins\Runtime\HairStrands\Source\HairStrandsCore\Private\GroomAsset.cpp(1592):				GetDerivedDataCacheRef().Put(*DerivedDataKey, DerivedData, GetPathName());
D:\svn\UnrealEngine\Engine\Plugins\Runtime\HairStrands\Source\HairStrandsCore\Private\GroomAsset.cpp(1636):		GetDerivedDataCacheRef().Put(*DerivedDataKey, DerivedData, GetPathName());
D:\svn\UnrealEngine\Engine\Plugins\Runtime\HairStrands\Source\HairStrandsCore\Private\GroomAsset.cpp(1708):		GetDerivedDataCacheRef().Put(*DerivedDataKey, DerivedData, GetPathName());
D:\svn\UnrealEngine\Engine\Plugins\Runtime\HairStrands\Source\HairStrandsCore\Private\GroomBindingAsset.cpp(653):					GetDerivedDataCacheRef().Put(*DerivedDataKey, DerivedData, GetPathName());
D:\svn\UnrealEngine\Engine\Source\Developer\PakFileUtilities\Private\PakFileUtilities.cpp(877):		GetDerivedDataCacheRef().Put(*DDCKey, GetData, InFile.Dest);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\AudioDerivedData.cpp(243):	GetDerivedDataCacheRef().Put(*DerivedDataKey, RawDerivedData, SoundWaveName);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\DistanceFieldAtlas.cpp(1289):				GetDerivedDataCacheRef().Put(*Task->DDCKey, DerivedData, Task->StaticMesh->GetPathName());
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Materials\MaterialShader.cpp(1172):	GetDerivedDataCacheRef().Put(*GetMaterialShaderMapKeyString(ShaderMapId, GetShaderPlatform(), TargetPlatform), SaveData, FStringView(GetFriendlyName()));
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Particles\ParticleModules.cpp(1322):		GetDerivedDataCacheRef().Put(*KeyString, Data, GetPathName());
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Particles\SubUVAnimation.cpp(123):		GetDerivedDataCacheRef().Put(*KeyString, Data, GetPathName());
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\ShaderCompiler\ShaderCompiler.cpp(4813):			GetDerivedDataCacheRef().Put(*GetGlobalShaderMapKeyString(ShaderMapId, Platform, TargetPlatform, ShaderFilenameDependencies.Value), SaveData, TEXT("GlobalShaderMap"_SV));
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\SkeletalMeshRenderData.cpp(359):			GetDerivedDataCacheRef().Put(*BuiltDerivedDataKey, DerivedData, Owner->GetPathName());
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\SoundWave.cpp(202):	GetDerivedDataCacheRef().Put(*InDerivedDataKey, DerivedData, SoundWaveName);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\StaticMesh.cpp(2827):				GetDerivedDataCacheRef().Put(*DerivedDataKey, DerivedData, Owner->GetPathName());
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\StaticMesh.cpp(4577):						GetDerivedDataCacheRef().Put(*MeshDataKey, DerivedData, GetPathName());
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Texture2D.cpp(171):	GetDerivedDataCacheRef().Put(*InDerivedDataKey, DerivedData, TextureName);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\TextureDerivedData.cpp(763):	GetDerivedDataCacheRef().Put(*DerivedDataKey, RawDerivedData, TextureName, /*bPutEvenIfExists*/ true);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\VT\VirtualTextureBuiltData.cpp(277):	GetDerivedDataCacheRef().Put(*InDerivedDataKey, DerivedData, TextureName);
D:\svn\UnrealEngine\Engine\Source\Runtime\Landscape\Private\Landscape.cpp(3741):	GetDerivedDataCacheRef().Put(*GetDDCKeyString(StateId), Bytes, Component->GetPathName());
D:\svn\UnrealEngine\Engine\Source\Runtime\Landscape\Private\LandscapeCollision.cpp(1057):			GetDerivedDataCacheRef().Put(*GetHFDDCKeyString(Format, bUseDefMaterial, HeightfieldGuid, InOutMaterials), OutCookedData, GetPathName());
D:\svn\UnrealEngine\Engine\Source\Runtime\Landscape\Private\LandscapeCollision.cpp(1283):			GetDerivedDataCacheRef().Put(*GetHFDDCKeyString(Format, bUseDefMaterial, MeshGuid, InOutMaterials), OutCookedData, GetPathName());

 

读取DDC缓存

/*** UnrealEngine\Engine\Source\Runtime\Engine\Private\Streaming\Texture2DStreamIn_DDC.cpp ***/
#if WITH_EDITORONLY_DATA
// ... ...
void FTexture2DStreamIn_DDC::DoLoadNewMipsFromDDC(const FContext& Context)
{
    if (Context.Texture && Context.Resource)
    {
        for (int32 MipIndex = PendingFirstLODIdx; MipIndex < CurrentFirstLODIdx && !IsCancelled(); ++MipIndex)
        {
            const FTexture2DMipMap& MipMap = *Context.MipsView[MipIndex];
            check(MipData[MipIndex]);

            if (!MipMap.DerivedDataKey.IsEmpty())
            {
                // The overhead of doing 2 copy of each mip data (from GetSynchronous() and FMemoryReader) in hidden by other texture DDC ops happening at the same time.
                TArray<uint8> DerivedMipData;
                bool bDDCValid = true;

                const uint32 Handle = DDCHandles[MipIndex];
                if (Handle)
                {
                    bDDCValid = GetDerivedDataCacheRef().GetAsynchronousResults(Handle, DerivedMipData);
                    DDCHandles[MipIndex] = 0;
                }
                else
                {
                    bDDCValid = GetDerivedDataCacheRef().GetSynchronous(*MipMap.DerivedDataKey, DerivedMipData, Context.Texture->GetPathName());
                }

                if (bDDCValid)                
                {
                    const int32 ExpectedMipSize = CalcTextureMipMapSize(MipMap.SizeX, MipMap.SizeY, Context.Resource->GetPixelFormat(), 0);
                    FMemoryReader Ar(DerivedMipData, true);

                    int32 MipSize = 0;
                    Ar << MipSize;

                    if (MipSize == ExpectedMipSize)
                    {
                        Ar.Serialize(MipData[MipIndex], MipSize);
                    }
                    else
                    {
                        UE_LOG(LogTexture, Error, TEXT("DDC mip size (%d) not as expected."), MipIndex);
                        MarkAsCancelled();
                    }
                }
                else
                {
                    // UE_LOG(LogTexture, Warning, TEXT("Failed to stream mip data from the derived data cache for %s. Streaming mips will be recached."), Context.Texture->GetPathName() );
                    MarkAsCancelled();
                }
            }
            else
            {
                UE_LOG(LogTexture, Error, TEXT("DDC key missing."));
                MarkAsCancelled();
            }
        }
        FPlatformMisc::MemoryBarrier();
    }
}

#endif

 

步骤如下:

① 传入Key调用GetDerivedDataCacheRef.GetSynchronous()同步或GetDerivedDataCacheRef().GetAsynchronous()异步获取DDC数据。

② 得到TArray<uint8> DerivedMipData二进制数据块后,将这个数组传入到FMemoryWriter就可以反序列化出数据。

引擎中调用GetDerivedDataCacheRef().GetSynchronous同步函数的地方:

D:\svn\UnrealEngine\Engine\Plugins\Compositing\OpenColorIO\Source\OpenColorIO\Private\OpenColorIOShaderMap.cpp(312):			if (GetDerivedDataCacheRef().GetSynchronous(*DataKey, CachedData, InColorTransform->GetFriendlyName()))
D:\svn\UnrealEngine\Engine\Plugins\FX\Niagara\Source\NiagaraShader\Private\NiagaraShader.cpp(584):			if (GetDerivedDataCacheRef().GetSynchronous(*DataKey, CachedData, Script->GetFriendlyName()))
D:\svn\UnrealEngine\Engine\Plugins\FX\Niagara\Source\Niagara\Private\NiagaraScript.cpp(1771):		if (GetDerivedDataCacheRef().GetSynchronous(*GetNiagaraDDCKeyString(), OutData, GetPathName()))
D:\svn\UnrealEngine\Engine\Plugins\FX\Niagara\Source\Niagara\Private\NiagaraSpriteRendererProperties.cpp(545):		if (GetDerivedDataCacheRef().GetSynchronous(*KeyString, Data, GetPathName()))
D:\svn\UnrealEngine\Engine\Plugins\FX\Niagara\Source\Niagara\Private\NiagaraSystem.cpp(1837):	if (GetDerivedDataCacheRef().GetSynchronous(*ScriptPair.CompiledScript->GetNiagaraDDCKeyString(), Data, GetPathName()))
D:\svn\UnrealEngine\Engine\Plugins\Runtime\HairStrands\Source\HairStrandsCore\Private\GroomAsset.cpp(1523):	if (GetDerivedDataCacheRef().GetSynchronous(*DerivedDataKey, DerivedData, GetPathName()))
D:\svn\UnrealEngine\Engine\Plugins\Runtime\HairStrands\Source\HairStrandsCore\Private\GroomAsset.cpp(1617):	if (GetDerivedDataCacheRef().GetSynchronous(*DerivedDataKey, DerivedData, GetPathName()))
D:\svn\UnrealEngine\Engine\Plugins\Runtime\HairStrands\Source\HairStrandsCore\Private\GroomAsset.cpp(1688):	if (GetDerivedDataCacheRef().GetSynchronous(*DerivedDataKey, DerivedData, GetPathName()))
D:\svn\UnrealEngine\Engine\Plugins\Runtime\HairStrands\Source\HairStrandsCore\Private\GroomBindingAsset.cpp(614):		if (GetDerivedDataCacheRef().GetSynchronous(*DerivedDataKey, DerivedData, GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Developer\Apple\MetalShaderFormat\Private\MetalShaderCompiler.cpp(1009):			bSucceeded = GetDerivedDataCacheRef().GetSynchronous(BytecodeCooker, OutData, &bDataWasBuilt);
D:\svn\UnrealEngine\Engine\Source\Developer\Apple\MetalShaderFormat\Private\MetalShaderCompiler.cpp(1595):	bool bCompiled = GetDerivedDataCacheRef().GetSynchronous(Cooker, OutData, &bDataWasBuilt) && OutData.Num();
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Animation\AnimCompressionDerivedData.cpp(331):		GetDerivedDataCacheRef().GetSynchronous(&Compressor, OutData);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Animation\AnimSequence.cpp(2323):		if (!bSkipDDC && GetDerivedDataCacheRef().GetSynchronous(*FinalDDCKey, OutData, AnimCompressor->GetDebugContextString()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Animation\AnimSequence.cpp(2353):					const bool bSuccess = GetDerivedDataCacheRef().GetSynchronous(AnimCompressor, OutData, &bBuilt);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Animation\AnimStreamable.cpp(632):		if (!bSkipDDC && GetDerivedDataCacheRef().GetSynchronous(*FinalDDCKey, OutData, GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Animation\AnimStreamable.cpp(686):				const bool bSuccess = GetDerivedDataCacheRef().GetSynchronous(AnimCompressor, OutData, &bBuilt);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\AudioDerivedData.cpp(533):		if (!bForceRebuild && GetDerivedDataCacheRef().GetSynchronous(*DerivedData->DerivedDataKey, RawDerivedData, SoundWave.GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\AudioStreaming.cpp(71):	if (GetDerivedDataCacheRef().GetSynchronous(*DerivedDataKey, DerivedChunkData, TEXT("Unknown Audio"_SV)))
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\DistanceFieldAtlas.cpp(915):	if (GetDerivedDataCacheRef().GetSynchronous(*InDDCKey, DerivedData, Mesh->GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Materials\MaterialShader.cpp(1138):			if (CheckCache && GetDerivedDataCacheRef().GetSynchronous(*DataKey, CachedData, Material->GetFriendlyName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Particles\ParticleModules.cpp(1308):	if (GetDerivedDataCacheRef().GetSynchronous(*KeyString, Data, GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Particles\SubUVAnimation.cpp(109):	if (GetDerivedDataCacheRef().GetSynchronous(*KeyString, Data, GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\PhysicsEngine\BodySetup.cpp(1482):			bDDCHit = GetDerivedDataCacheRef().GetSynchronous(&InBuilder, OutData, &bDataWasBuilt);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\SkeletalMeshRenderData.cpp(153):		if (GetDerivedDataCacheRef().GetSynchronous(*DerivedDataKey, DerivedData, Owner->GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\SoundWave.cpp(967):				bGetSuccessful = GetDerivedDataCacheRef().GetSynchronous(DeriveAudioData, OutData, &bDataWasBuilt);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\StaticMesh.cpp(2747):		if (GetDerivedDataCacheRef().GetSynchronous(*DerivedDataKey, DerivedData, Owner->GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\StaticMesh.cpp(4286):		if (GetDerivedDataCacheRef().GetSynchronous(*MeshDataKey, DerivedData, GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Streaming\Texture2DStreamIn_DDC.cpp(181):					bDDCValid = GetDerivedDataCacheRef().GetSynchronous(*MipMap.DerivedDataKey, DerivedMipData, Context.Texture->GetPathName());
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\TextureDerivedDataTask.cpp(446):	if (!bForceRebuild && GetDerivedDataCacheRef().GetSynchronous(*DerivedData->DerivedDataKey, RawDerivedData, Texture.GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Experimental\GeometryCollectionEngine\Private\GeometryCollection\GeometryCollectionObject.cpp(417):			const bool bSuccess = GetDerivedDataCacheRef().GetSynchronous(GeometryCollectionCooker, DDCData, &bBuilt);
D:\svn\UnrealEngine\Engine\Source\Runtime\Landscape\Private\Landscape.cpp(3726):	if (GetDerivedDataCacheRef().GetSynchronous(*GetDDCKeyString(StateId), Bytes, Component->GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Landscape\Private\LandscapeCollision.cpp(884):			if (GetDerivedDataCacheRef().GetSynchronous(*DDCKey, OutCookedData, GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\Landscape\Private\LandscapeCollision.cpp(1114):			if (GetDerivedDataCacheRef().GetSynchronous(*DDCKey, OutCookedData, GetPathName()))
D:\svn\UnrealEngine\Engine\Source\Runtime\NavigationSystem\Private\NavCollision.cpp(553):			if (GetDerivedDataCacheRef().GetSynchronous(DerivedNavCollisionData, OutData, &bDataWasBuilt))

引擎中调用GetDerivedDataCacheRef().GetAsynchronous异步函数的地方:

D:\svn\UnrealEngine\Engine\Plugins\Editor\AssetSearch\Source\Private\AssetSearchManager.cpp(759):        DDCRequest.DDCHandle = GetDerivedDataCacheRef().GetAsynchronous(*DDCRequest.DDCKey, DDCRequest.AssetData.ObjectPath.ToString());
D:\svn\UnrealEngine\Engine\Source\Developer\Apple\MetalShaderFormat\Private\MetalShaderCompiler.cpp(951):                DebugInfoHandle = GetDerivedDataCacheRef().GetAsynchronous(DebugInfoCooker);
D:\svn\UnrealEngine\Engine\Source\Developer\PakFileUtilities\Private\PakFileUtilities.cpp(804):            int32 AsyncHandle = GetDerivedDataCacheRef().GetAsynchronous(*DDCKey, InFile.Dest);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Animation\AnimCompressionDerivedData.cpp(344):    uint32 AsyncHandle = GetDerivedDataCacheRef().GetAsynchronous(&Compressor);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\ShaderCompiler\ShaderCompiler.cpp(5050):                AsyncDDCRequestHandles[HandleIndex] = GetDerivedDataCacheRef().GetAsynchronous(*DataKey, TEXT("GlobalShaderMap"_SV));
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\SoundWave.cpp(916):            uint32 GetHandle = GetDerivedDataCacheRef().GetAsynchronous(DeriveAudioData);
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Streaming\Texture2DMipDataProvider_DDC.cpp(36):                DDCHandles[MipIndex] = GetDerivedDataCacheRef().GetAsynchronous(*OwnerMip.DerivedDataKey, Context.Texture->GetPathName());
D:\svn\UnrealEngine\Engine\Source\Runtime\Engine\Private\Streaming\Texture2DStreamIn_DDC.cpp(126):                DDCHandles[MipIndex] = GetDerivedDataCacheRef().GetAsynchronous(*MipMap.DerivedDataKey, Context.Texture->GetPathName());
D:\svn\UnrealEngine\Engine\Source\Runtime\Landscape\Private\LandscapeCollision.cpp(743):            uint32 Handle = GetDerivedDataCacheRef().GetAsynchronous(*Key, GetPathName());

 

编辑器中打开地图资产没有DDC或失效时,会重新生成DDC

LogMaterial: Display: |MaterialShared.cpp:2331|Missing cached shader map for material M_Env_Grass, compiling. 
LogDistanceFieldAtlas: Warning: |DistanceFieldAtlas.cpp:311|Loaded empty distance field data from DDC SM_Env_Plants_BurnGrass
LogMaterial: Display: |MaterialShared.cpp:2331|Missing cached shader map for material M_Env_Tree, compiling. 
LogMaterial: Display: |MaterialShared.cpp:2331|Missing cached shader map for material M_Env_Tree, compiling. 
LogMaterial: Display: |MaterialShared.cpp:2331|Missing cached shader map for material M_Env_Tree, compiling. 
LogMaterial: Display: |MaterialShared.cpp:2331|Missing cached shader map for material M_Env_Impostor, compiling. 
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine05_A03_2_Billboard_IBA(AutoDXT、2048X2048)
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine05_A03_2_Billboard_IND(AutoDXT、2048X2048)
LogMaterial: Display: |MaterialShared.cpp:2331|Missing cached shader map for material M_Env_Tree, compiling. 
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine05_A01_Leaf_BR(AutoDXT、2048X2048)
LogMaterial: Display: |MaterialShared.cpp:2331|Missing cached shader map for material M_Env_Tree, compiling. 
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine01_A02_Leaf_BR(AutoDXT、2048X2048)
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine05_A02_Billboard_IBA(AutoDXT、4096X4096)
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine05_A02_Billboard_IND(AutoDXT、4096X4096)
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine01_A01_Trunk_B(AutoDXT、2048X2048)
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine01_A01_Trunk_ORN(BC7、2048X2048)
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine05_A01_Billboard_IBA(AutoDXT、4096X4096)
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine05_A01_Billboard_IND(AutoDXT、4096X4096)
LogMaterial: Display: |MaterialShared.cpp:2331|Missing cached shader map for material M_Env_Tree, compiling. 
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine04_A02_Billboard_IBA(AutoDXT、2048X2048)
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine04_A02_Billboard_IND(AutoDXT、2048X2048)
LogStaticMesh: |StaticMesh.cpp:2628|正在构建静态网格体SM_Env_Tree_Pine04……
LogStaticMeshBuilder: Display: |StaticMeshBuilder.cpp:361|Finished staticMesh build in 0.4s - StaticMesh /Game/SM_Env_Tree_Pine04.SM_Env_Tree_Pine04
LogStaticMesh: |StaticMesh.cpp:3033|Built static mesh [0.42s] /Game/SM_Env_Tree_Pine04.SM_Env_Tree_Pine04
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine_Field01_Leaf_BR(AutoDXT、4096X4096)
LogTexture: Warning: |TextureDerivedDataTask.cpp:495|The data retrieved from the derived data cache for the texture Texture2D /Game/T_Env_Tree_Pine_Field01_Leaf_ORN.T_Env_Tree_Pine_Field01_Leaf_ORN was invalid. The texture will be rebuild.
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine_Field01_Leaf_ORN(BC7、4096X4096)
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine_Field01_Leaf_AT(AutoDXT、4096X4096)
LogMeshUtilities: Display: |MeshDistanceFieldUtilities.cpp:630|Finished distance field build in 5.4s - 63x63x63 sparse distance field, 0.4Mb total, 0.0Mb always loaded, 77% occupied, 3840 triangles, SM_Env_Tree_Pine04
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine04_A01_Billboard_IBA(AutoDXT、2048X2048)
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine04_A01_Billboard_IND(AutoDXT、2048X2048)
LogStaticMesh: |StaticMesh.cpp:2628|正在构建静态网格体SM_Env_Pine_Field04……
LogStaticMeshBuilder: Display: |StaticMeshBuilder.cpp:361|Finished staticMesh build in 0.5s - StaticMesh /Game/SM_Env_Pine_Field04.SM_Env_Pine_Field04
LogStaticMesh: |StaticMesh.cpp:3033|Built static mesh [0.55s] /Game/SM_Env_Pine_Field04.SM_Env_Pine_Field04
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine06_A01_Trunk_B(AutoDXT、2048X2048)
LogTexture: Warning: |TextureDerivedDataTask.cpp:495|The data retrieved from the derived data cache for the texture Texture2D /Game/T_Env_Tree_Pine06_A01_Trunk_ORN.T_Env_Tree_Pine06_A01_Trunk_ORN was invalid. The texture will be rebuild.
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine06_A01_Trunk_ORN(BC7、2048X2048)
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine06_A02_Billboard_IBA(AutoDXT、2048X2048)
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine06_A02_Billboard_IND(AutoDXT、2048X2048)
LogMeshUtilities: Display: |MeshDistanceFieldUtilities.cpp:630|Finished distance field build in 2.8s - 63x63x63 sparse distance field, 0.3Mb total, 0.0Mb always loaded, 71% occupied, 4007 triangles, SM_Env_Pine_Field04
LogTexture: Display: |TextureDerivedDataTask.cpp:44|正在编译纹理:T_Env_Tree_Pine_Field_02_A01_Leaf_BR(AutoDXT、2048X2048)
。。。 。。。 。。。 。。。
LogTemp: Display: |NiagaraShared.cpp:590|Loading shader for Niagara script NS_Hit_Melee_Fruit_QL/Water04/ParticleGPUComputeScript from DDC failed. Shader needs recompile.
LogNiagaraShaderCompiler: Verbose: |NiagaraShaderCompilationManager.cpp:48|Adding niagara gpu shader compile job... NS_Hit_Melee_Fruit_QL/Water04/ParticleGPUComputeScript/None/FNiagaraShader/0
LogTemp: Display: |NiagaraShared.cpp:590|Loading shader for Niagara script NS_Hit_Melee_Ceramic_QL/Smoke03/ParticleGPUComputeScript from DDC failed. Shader needs recompile.
LogNiagaraShaderCompiler: Verbose: |NiagaraShaderCompilationManager.cpp:48|Adding niagara gpu shader compile job... NS_Hit_Melee_Ceramic_QL/Smoke03/ParticleGPUComputeScript/None/FNiagaraShader/0
LogTemp: Display: |NiagaraShared.cpp:590|Loading shader for Niagara script NS_Hit_Melee_Ceramic_QL/Smoke01/ParticleGPUComputeScript from DDC failed. Shader needs recompile.
LogNiagaraShaderCompiler: Verbose: |NiagaraShaderCompilationManager.cpp:48|Adding niagara gpu shader compile job... NS_Hit_Melee_Ceramic_QL/Smoke01/ParticleGPUComputeScript/None/FNiagaraShader/0
LogTemp: Display: |NiagaraShared.cpp:590|Loading shader for Niagara script NS_Hit_Melee_Ceramic_QL/Stone02/ParticleGPUComputeScript from DDC failed. Shader needs recompile.
LogNiagaraShaderCompiler: Verbose: |NiagaraShaderCompilationManager.cpp:48|Adding niagara gpu shader compile job... NS_Hit_Melee_Ceramic_QL/Stone02/ParticleGPUComputeScript/None/FNiagaraShader/0
LogAnimationCompression: |AnimCompressionDerivedData.cpp:55|Building Anim DDC data for AnimSequence /Game/AO/TP_Prone_AO_RU.TP_Prone_AO_RU
LogAnimationCompression: |AnimCompressionDerivedData.cpp:55|Building Anim DDC data for AnimSequence /Game/AO/TP_Prone_AO_RD.TP_Prone_AO_RD 

 

DDC Key

SkeletalMesh

MaterialShaderMap

#if WITH_EDITOR
/** Creates a string key for the derived data cache given a shader map id. */
static FString GetMaterialShaderMapKeyString(const FMaterialShaderMapId& ShaderMapId, EShaderPlatform Platform, const ITargetPlatform* TargetPlatform)
{
    FName Format = LegacyShaderPlatformToShaderFormat(Platform);
    FString ShaderMapKeyString = Format.ToString() + TEXT("_") + FString(FString::FromInt(GetTargetPlatformManagerRef().ShaderFormatVersion(Format))) + TEXT("_");

    ShaderMapAppendKeyString(Platform, ShaderMapKeyString);
    ShaderMapId.AppendKeyString(ShaderMapKeyString);
    FMaterialAttributeDefinitionMap::AppendDDCKeyString(ShaderMapKeyString);
    return FDerivedDataCacheInterface::BuildCacheKey(TEXT("MATSM"), MATERIALSHADERMAP_DERIVEDDATA_VER, *ShaderMapKeyString);
}
#endif // WITH_EDITOR

 

常见DDC类型和Key:

类型 VER Key
 GlobalShaderMap GLOBALSHADERMAP_DERIVEDDATA_VER

GSM_0875A72E4CE141409B09970FDA2C8978_PCD3D_ES31_8__BC5N_NOCCBN_NOIRIS_DEV_SL_P__62AA06E30D4FCE7BA88145D25DF42B52B74EAE5E
GSM_0875A72E4CE141409B09970FDA2C8978_PCD3D_SM5_8__BC5N_NOCCBN_NOIRIS_DEV_SL_PR__90BE85B7944C9B3EEE7C522B54169F803BFB6306
GSM_0875A72E4CE141409B09970FDA2C8978_SF_VULKAN_ES31_2334__BC5N_NOCCBN_NOIRIS_D__B8C489DD85A5C021DF73A78207EB1B5BEB44F486
GSM_0875A72E4CE141409B09970FDA2C8978_SF_VULKAN_SM5_2334__BC5N_NOCCBN_NOIRIS_DE__5460C9AC126722ACCCDC7067311E97421893C9D7

MaterialShaderMap MATERIALSHADERMAP_DERIVEDDATA_VER

MATSM_01BF92A5B29A44BE8A839E4251DA49D7_PCD3D_ES31_8__BC5N_NOCCBN_NOIRIS_DEV_SL__5522BBD10C8F3AF575C4915882212CE8BC12804E
MATSM_01BF92A5B29A44BE8A839E4251DA49D7_PCD3D_SM5_8__BC5N_NOCCBN_NOIRIS_DEV_SL___6D84A87569E35232A9271EC291D9AB716BD7AA6F
MATSM_01BF92A5B29A44BE8A839E4251DA49D7_SF_VULKAN_ES31_2334__BC5N_NOCCBN_NOIRIS__E974F44CF4168ED0572AC64780814F1AE78C0094
MATSM_01BF92A5B29A44BE8A839E4251DA49D7_SF_VULKAN_SM5_2334__BC5N_NOCCBN_NOIRIS___B06A61B1A7EE9B8E1815A31384A55860B15617A4

NiagaraShaderMap NIAGARASHADERMAP_DERIVEDDATA_VER

NIAGARASM_34571112$2DFD93$2D4794$2DB613$2DCBEEF253AF84_SF_VULKAN_SM5_2334_2C74__D797E70ECED647A0932952F7D1E3185DC7BF40A7

 Texture TEXTURE_DERIVEDDATA_VER  

TEXTURE_2D528FAD496A42E180956107C6FFBD67_BGRA8_748D4F054C3030AC8888BEA1993C93A__00BA9B04CB4EECFA96840E6DF67034D5C203C3FA
TEXTURE_2D528FAD496A42E180956107C6FFBD67_RGBA16F_6086FCE04C15F5EA169ADAA28C8CC__6DE61B505F00035BA11E3C73DD72450D6E178AE2
TEXTURE_2D528FAD496A42E180956107C6FFBD67_G8_BBFD25CA40EF4EA010BE5E964BCA0654_0__2D4B1A6DFB0AE377D9D49F0590A8654D678C0E11
TEXTURE_2D528FAD496A42E180956107C6FFBD67_AUTODXT_5662EE7848FCABBBB6A3DAB72817A__9248C40F8691D10C0FB41612A45003AB24478A9A
TEXTURE_2D528FAD496A42E180956107C6FFBD67_DXT1_2C1F8AFB4E38620171A5D3ADF1428090__088610C266311B177A9F0A125C60059B4DF3EB01
TEXTURE_2D528FAD496A42E180956107C6FFBD67_BC5_BDBEA3F14397ABAB02BEE19D6B07D1DD___F38F51AB157C6307DB5E311D124967880E4F224D
TEXTURE_2D528FAD496A42E180956107C6FFBD67_BC7_4_5E42B61E4603AF48A6C0AD830769BFD__AE9F2501645A1078520479DF1C2E8983D8C8C5FB

 StaticMesh MESHDATAKEY_STATICMESH_DERIVEDDATA_VER  

STATICMESH_14FB7BD55BAA4C14BCDC7C2AD14BCE50_228332BAE0224DD294E232B87D83948FQU__70371012EEFD9BD9DC1FA71BF73A89D39856A005
MESHDATAKEY_STATICMESH_940C97E946F545F5834146D0D58828BF_5B5FBB3C90C74720BE02B0A2013BC62C

StaticMesh Distance Field(sdf) 注:ue5才有 DISTANCEFIELD_DERIVEDDATA_VER DIST_STATICMESH_5CD3B43153AC41619FDC648B68EA1203_228332BAE0224DD294E232B87D839__46E12BFED501542B1582701807991CF16A4CB525
 SkeletalMesh SKELETALMESH_DERIVEDDATA_VER SKELETALMESH_7E02F227B356416F95DACCC99F757AD7_C62F737348494F34916095A57FF2EBF8__51148C90662BD28C223DB6E6B11DA54D5AEC97B9
 PhysX LANDSCAPE_COLLISION_DERIVEDDATA_VER  

PHYSX_A08609EF508D460CBB6710540B03AACF_PHYSXPC_2681797E4C47F22E025D90A9BEAC2E4F__1_1_0_0_2_13315_0
PHYSX_A08609EF508D460CBB6710540B03AACF_PHYSXPC_EC8220CE406302663770B7B13FC8031__7F862994B073A6EF08858CBE3DAFE4D0FD51DFE8
PHYSX_PHYSXPC_VIS_CC58B9FA08AD47E3BF06976E60B693C3_3BC8F3D84D612C296C0E2CABD18BABE2
PHYSX_PHYSXPC_FULL_CC58B9FA08AD47E3BF06976E60B693C3_1CCCC11297CFA39D531A9D88A8EBA593

NavCollision   NAVCOLLISION_2983273BE5D6425391E8C0B3D1C0A55D_NAVCOLLISION_X_A40CFA28462BC7B14__84A663C6DA1450FB2CA08FD49BD4BB01532C8B31
 AnimSequence FDerivedDataAnimationCompression::GetVersionString()  ANIMSEQ_D8E26604610C43A4A2DCB8A690657705_0_3214C687B983834BB01F3466EF5B997F945__B53996C771A9C86F22CAE5417F8E3F80B1B8D8F0
 Particle(SubUV) SUBUV_DERIVEDDATA_VER  SUBUV__67E9AF86DF8B4D8E97B7A614A73CD4BF_44DD8757494DFEE554977DB47EF4041F_6_6_1_0$2E001000_V2
 NiagaraScript NIAGARASCRIPT_DERIVEDDATA_VER  NIAGARASCRIPTDERIVEDDATA_179023FDDDD444DE97F61296909C2990_2_17_6FC3CEC14F14B25__A626814BD4853484E8CAFEEF477D45777FC2788D
 。。。 。。。   。。。。。。

 

DDC缓存目录结构

 

本地DDC

本地DDC默认位置

Installed版本:

C:/Users/<user>/AppData/Local/UnrealEngine/Common/DerivedDataCache

源码版本:

D:\svn\UnrealEngine\Engine\DerivedDataCache   // 本地DDC: 项目的派生数据。

 

iOS下Metal shader的DDC在:

UnrealEngine/Engine/Programs/ShaderCompileWorker/DDC

 

自定义本地DDC位置

可以通过在 DefaultEngine.ini 中添加一个部分来更改本地缓存位置。 

以下示例将本地缓存移动到 Project\DerivedDataCache 下,而不是默认位置。

Installed版本:

[InstalledDerivedDataBackendGraph]
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=34, FoldersToClean=-1, Path="%GAMEDIR%DerivedDataCache")

源码版本:

[DerivedDataBackendGraph]
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=34, FoldersToClean=-1, Path="%GAMEDIR%DerivedDataCache")

注1:也可通过点击编辑器菜单Editor -- Editor Preferences...,在弹出面板的General -- Global标签中,设置Local Derived Data Cache路径

注2:将 UE-LocalDataCachePath(在Mac/Linux上为 UE_LocalDataCachePath)的环境变量设置为要使用的文件夹。如:setx UE-LocalDataCachePath d:\DDC

 

共享DDC

将DDC缓存文件放在可读写的高速网络共享磁盘上,让整个团队都能访问到,以避免每个人都需要在本地单独生成DDC造成的时间浪费和存储的空间浪费。

下文按照最优到最次的顺序列出了推荐方式。

  1. 在DefaultEngine.ini中覆盖你的项目设置(如下图所示),该覆盖项会将路径设置为团队的有效位置。

    [DerivedDataBackendGraph] 
    Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=10, MaxFileChecksPerSec=1, ConsiderSlowAt=70, PromptIfMissing=false, Path=\\YourCompanyServer\DDC, EnvPathOverride=UE-SharedDataCachePath, EditorOverrideSetting=SharedDerivedDataCache)
  2. 将 UE-SharedDataCachePath(在Mac/Linux上为 UE_SharedDataCachePath)的环境变量设置为要使用的文件夹。注:永远不要将 UE-SharedDataCachePath 设置为本地路径!这样做意味着你将同时占用计算机上的本地缓存和共享缓存——两倍的磁盘空间占用量,且收益为零!

    net use \\YourCompanyServer\DDC  "password" /user:"admin" /persistent:yes  // windows上映射网络驱动器
    setx UE-SharedDataCachePath \\YourCompanyServer\DDC  // 设置用户环境变量UE-SharedDataCachePath为\\YourCompanyServer\DDC
  3. 通过点击编辑器菜单Editor -- Editor Preferences...,在弹出面板的General -- Global标签中,设置Shared Derived Data Cache路径.

禁用共享DDC

当网速慢的时候,访问DDC数据比自己生成DDC数据花费的时间更长,这个时候可以禁用掉共享DDC。

请使用以下方法之一:

  • 在命令行上传递-ddc=noshared

  • 将环境变量设置为UE-SharedDataCachePath=None(在Mac/Linux上:UE_SharedDataCachePath=None

构建DDC缓存

可以在构建机上每夜定时执行此操作,以确保DDC缓存始终处于就绪状态,提升团队的工作效率。

D:\svn\UnrealEngine\Engine\Binaries\Win64\UE4Editor.exe MyGame -run=DerivedDataCache -fill

 

DDC Pak

如果你从Epic Games商店下载虚幻引擎,则该引擎将附带DDC Pak (.ddp)。

DDC Pak包含所有引擎内容的派生数据,因此你无需编译着色器等就可以开始工作。同样,出于相同原因,一些示例也随DDC Pak一起提供。

引擎DDC Pak:EngineDir/DerivedDataCache/Compressed.ddp
项目DDC Pak:ProjectDir/DerivedDataCache/Compressed.ddp

创建DDC Pak

引擎DDC Pak:UE4Editor.exe -run=DerivedDataCache -fill -DDC=CreatePak
项目DDC Pak:UE4Editor.exe MyGame -run=DerivedDataCache -fill -DDC=CreatePak

注1:这会在UE4\MyGame\DerivedDataCache目录中创建DDC.ddp文件

注2:引擎将自动检测和使用.ddp文件

 

启动DDC

加载到内存中以便提升启动时间的启动文件

Installed版本:

C:/Users/<user>/AppData/Local/UnrealEngine/4.26/DerivedDataCache/Boot.ddc 

源码版本:

D:\svn\MyGame\DerivedDataCache

 

DDC配置

DDC配置存放在UnrealEngine\Engine\Config\BaseEngine.ini相关的标签中,可在项目的DefaultEngine.ini中改写这些配置。

; Derived Data backend graphs
; Each of the below backend graph sections contains a set of nodes used to create derived data backed graph.
; DerivedDataBackendGraph is the default graph for source builds, and InstalledDerivedDataBackendGraph is the default for installed builds.
; Others can be specified othe command line using:
;     -DDC=GraphSectionName     (for example: -DDC=VerifyDerivedDataBackendGraph)
; Each graph should start with 'Root' node. Names of all the other nodes are not predefined.
; Supported node types are: KeyLength, AsyncPut, Hierarchical, Boot, Filesystem, ReadPak, WritePak, Verify
; The order nodes are define in is not relevant
; Filesystem backends can be disabled by setting the corresponding environment variable to 'None'. E.g UE-SharedDataCachePath=None

[DerivedDataBackendGraph]
; Keep files for at least 7 days  保留文件至少7天
MinimumDaysToKeepFile=7
; Configure the root entry. It uses a KeyLength node to hash long strings then passes requests to AsyncPut   配置根条目。它使用KeyLength节点对长字符串进行哈希处理,然后将请求传递给AsyncPut
Root=(Type=KeyLength, Length=120, Inner=AsyncPut)
; Configure the AsyncPut entry. It uses an AsyncPut node that then passes requests to Hierarchy   配置AsyncPut条目。它使用AsyncPut节点,然后将请求传递到层级
AsyncPut=(Type=AsyncPut, Inner=Hierarchy)
; Configure the hierarchy entry. This uses multiple nodes that are used in order until a read is found (writes go to all writable entries) 配置层级条目。这将使用多个节点,这些节点按顺序使用,直到找到读取内容(写入将转到所有可写条目)
Hierarchy=(Type=Hierarchical, Inner=Boot, Inner=Pak, Inner=EnginePak, Inner=Local, Inner=Shared)
; Configure the Boot node. This holds 512MB of data used to speed up booting.  配置Boot节点。这将保留用于加速启动的512MB数据
Boot=(Type=Boot, Filename="%GAMEDIR%DerivedDataCache/Boot.ddc", MaxCacheSize=512) 
; Configure the local node. This is a filesystem node with the following set -  配置本地节点。这是具有以下设置的filesystem节点
;    Readonly: Can data be written to this layer  能否将数据写入此层
;    Clean: Perform a cleanup of old files on launch  在启动时执行旧文件清理
;    Flush: Nuke the DDC and start over   破坏DDC并重新开始
;    PurgeTransient: Don't keep transient data in the DDC  不要将瞬态数据保留在DDC中
;    DeleteUnused: Clean up old files (happens on a background thread)  清理旧文件(在后台线程上进行)  
;    UnusedFileAge: Age after which files are removed  文件删除前保留时长  注:文件访问时间和修改时间距离当前时间都超过该值时,则执行删除
;    FoldersToClean: Max number of folders to clean in a session. -1 = Unlimited  会话中要清除的最大文件夹数。-1 = 无限
;    PromptIfMissing: If the path does not exist a warning prompt will be shown. This is useful for shared DDCs where the mount may have dropped
;    ConsiderSlowAt: If access times are < this value in ms then some operations will be disabled to improve performance.
;    MaxFileChecksPerSec: How many files to check a second.  每秒检查的文件数量。
;    Path: path to use for the filesystem DDC   文件系统DDC使用的路径
;    EnvPathOverride: An environment variable that if set will be used instead of path. E.g. UE-LocalDataCachePath=d:\DDC. ('None' disables the DDC)  环境变量,如果设置,将用于代替路径。如UE-LocalDataCachePath=d:\DDC。( 'None' 会禁用DDC)
;    CommandLineOverride: A command line argument used in preference to the default / envvar setting. E.g. -SharedDataCachePath=\\someshare\folder  优先于默认值/envvar设置使用的命令行参数。如-SharedDataCachePath=\\someshare\folder
;    EditorOverrideSetting: Editor user setting that overrides the default/envvarcommand line values  覆盖默认/envvar/命令行值的编辑器用户设置
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=-1, PromptIfMissing=true, Path=%ENGINEDIR%DerivedDataCache, EnvPathOverride=UE-LocalDataCachePath, EditorOverrideSetting=LocalDerivedDataCache)
; Configure the shared DDC that is accessed after local. It's a filesystem DDC and the parameters are explained above   配置在本地之后访问的共享DDC。这是文件系统DDC,上面已经说明参数
Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=6, FoldersToClean=10, MaxFileChecksPerSec=1, ConsiderSlowAt=70, PromptIfMissing=false, Path=?EpicDDC, EnvPathOverride=UE-SharedDataCachePath, EditorOverrideSetting=SharedDerivedDataCache, CommandLineOverride=SharedDataCachePath)
; Configure an alternate shared DDC that is accessed after local. It's a filesystem DDC and the parameters are explained above  配置在本地之后访问的替代共享DDC。这是文件系统DDC,上面已经说明参数
AltShared=(Type=FileSystem, ReadOnly=true, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=8, FoldersToClean=10, MaxFileChecksPerSec=1, ConsiderSlowAt=70, PromptIfMissing=false, Path=?EpicDDC2, EnvPathOverride=UE-SharedDataCachePath2)
; Configure a Project Pak node. This is a pre-generated DDC data file for the project that can be distributed to reduce runtime fetches/generation  配置Project Pak节点。这是项目的预生成DDC数据文件,可以发布该文件以便减少运行时获取/生成
; See documentation for how to create a DDP via the DerivedDataCache commandlet   请参阅文档,了解如何通过DerivedDataCache Commandlet创建DDP
Pak=(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
; Configure a Project Pak node. This is a pre-generated DDC data file for the engine that can be distributed to reduce runtime fetches/generation  配置Project Pak节点。这是引擎的预生成DDC数据文件,可以分发该文件以便减少运行时获取/生成
EnginePak=(Type=ReadPak, Filename=%ENGINEDIR%DerivedDataCache/DDC.ddp)

[DerivedDataBackendGraph_Fill_Seattle]
MinimumDaysToKeepFile=7
Root=(Type=KeyLength, Length=120, Inner=AsyncPut)
AsyncPut=(Type=AsyncPut, Inner=Hierarchy)
Hierarchy=(Type=Hierarchical, Inner=Boot, Inner=Pak, Inner=EnginePak, Inner=Local, Inner=Seattle)
Boot=(Type=Boot, Filename="%GAMEDIR%DerivedDataCache/Boot.ddc", MaxCacheSize=512)
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=-1, Path=%ENGINEDIR%DerivedDataCache)
Seattle=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=8, FoldersToClean=10, MaxFileChecksPerSec=1, Path=?EpicSeaDDC, EnvPathOverride=UE-SharedDataCachePath_Seattle)
Pak=(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
EnginePak=(Type=ReadPak, Filename=%ENGINEDIR%DerivedDataCache/DDC.ddp)

[InstalledDerivedDataBackendGraph]
MinimumDaysToKeepFile=7
Root=(Type=KeyLength, Length=120, Inner=AsyncPut)
AsyncPut=(Type=AsyncPut, Inner=Hierarchy)
Hierarchy=(Type=Hierarchical, Inner=Boot, Inner=Pak, Inner=CompressedPak, Inner=EnginePak, Inner=EnterprisePak, Inner=Local, Inner=Shared)
Boot=(Type=Boot, Filename="%ENGINEUSERDIR%DerivedDataCache/Boot.ddc", MaxCacheSize=512)
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=-1, Path="%ENGINEVERSIONAGNOSTICUSERDIR%DerivedDataCache", EditorOverrideSetting=LocalDerivedDataCache)
Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=6, FoldersToClean=10, MaxFileChecksPerSec=1, Path=?EpicDDC, EnvPathOverride=UE-SharedDataCachePath, EditorOverrideSetting=SharedDerivedDataCache)
Pak=(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
CompressedPak=(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/Compressed.ddp", Compressed=true)
EnginePak=(Type=ReadPak, Filename=../../../Engine/DerivedDataCache/Compressed.ddp, Compressed=true)
EnterprisePak=(Type=ReadPak, Filename=../../../Enterprise/DerivedDataCache/Compressed.ddp, Compressed=true)

[NoShared]
MinimumDaysToKeepFile=7
Root=(Type=KeyLength, Length=120, Inner=AsyncPut)
AsyncPut=(Type=AsyncPut, Inner=Hierarchy)
Hierarchy=(Type=Hierarchical, Inner=Boot, Inner=Pak, Inner=Local)
Boot=(Type=Boot, Filename="%GAMEDIR%DerivedDataCache/Boot.ddc", MaxCacheSize=512)
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=-1, Path=%ENGINEDIR%DerivedDataCache, EnvPathOverride=UE-LocalDataCachePath)
Pak=(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")

[CreatePak]
MinimumDaysToKeepFile=7
Root=(Type=KeyLength, Length=120, Inner=AsyncPut)
AsyncPut=(Type=AsyncPut, Inner=Hierarchy)
Hierarchy=(Type=Hierarchical, Inner=Boot, Inner=PakWrite, Inner=PakRead, Inner=Local, Inner=Shared)
Boot=(Type=Boot, Filename="%GAMEDIR%DerivedDataCache/Boot.ddc", MaxCacheSize=512)
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=-1, Path=%ENGINEDIR%DerivedDataCache)
Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=false, UnusedFileAge=6, FoldersToClean=-1, Path=?EpicDDC, EnvPathOverride=UE-SharedDataCachePath)
AltShared=(Type=FileSystem, ReadOnly=true, Clean=false, Flush=false, DeleteUnused=false, UnusedFileAge=6, FoldersToClean=-1, Path=?EpicDDC2, EnvPathOverride=UE-SharedDataCachePath2)
PakRead=(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")
PakWrite=(Type=WritePak, Filename="%GAMEDIR%DerivedDataCache/DDC.ddp")

[CreateInstalledProjectPak]
MinimumDaysToKeepFile=7
Root=(Type=KeyLength, Length=120, Inner=AsyncPut)
AsyncPut=(Type=AsyncPut, Inner=Hierarchy)
Hierarchy=(Type=Hierarchical, Inner=Boot, Inner=EnginePak, Inner=PakWrite, Inner=PakRead, Inner=Local, Inner=Shared)
Boot=(Type=Boot, Filename="%GAMEDIR%DerivedDataCache/Boot.ddc", MaxCacheSize=512)
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=-1, Path=%ENGINEDIR%DerivedDataCache, EnvPathOverride=UE-LocalDataCachePath)
Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=false, UnusedFileAge=8, FoldersToClean=-1, Path=?EpicDDC, EnvPathOverride=UE-SharedDataCachePath, CommandLineOverride=SharedDataCachePath)
AltShared=(Type=FileSystem, ReadOnly=true, Clean=false, Flush=false, DeleteUnused=false, UnusedFileAge=8, FoldersToClean=-1, Path=?EpicDDC2, EnvPathOverride=UE-SharedDataCachePath2)
EnginePak=(Type=ReadPak, Filename=../../../Engine/DerivedDataCache/Compressed.ddp, Compressed=true)
PakRead=(Type=ReadPak, Filename="%GAMEDIR%DerivedDataCache/Compressed.ddp", Compressed=true)
PakWrite=(Type=WritePak, Filename="%GAMEDIR%DerivedDataCache/Compressed.ddp", Compressed=true)

[CreateInstalledEnginePak]
MinimumDaysToKeepFile=7
Root=(Type=KeyLength, Length=120, Inner=AsyncPut)
AsyncPut=(Type=AsyncPut, Inner=Hierarchy)
Hierarchy=(Type=Hierarchical, Inner=Boot, Inner=PakWrite, Inner=Local, Inner=Shared)
Boot=(Type=Boot, Filename="%GAMEDIR%DerivedDataCache/Boot.ddc", MaxCacheSize=512)
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=-1, Path=%ENGINEDIR%DerivedDataCache, EnvPathOverride=UE-LocalDataCachePath)
Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=false, UnusedFileAge=8, FoldersToClean=-1, Path=?EpicDDC, EnvPathOverride=UE-SharedDataCachePath)
AltShared=(Type=FileSystem, ReadOnly=true, Clean=false, Flush=false, DeleteUnused=false, UnusedFileAge=8, FoldersToClean=-1, Path=?EpicDDC2, EnvPathOverride=UE-SharedDataCachePath2)
PakWrite=(Type=WritePak, Filename=%ENGINEDIR%DerivedDataCache/Compressed.ddp, Compressed=true)

[CreateInstalledEnterprisePak]
MinimumDaysToKeepFile=7
Root=(Type=KeyLength, Length=120, Inner=AsyncPut)
AsyncPut=(Type=AsyncPut, Inner=Hierarchy)
Hierarchy=(Type=Hierarchical, Inner=Boot, Inner=PakWrite, Inner=Local, Inner=Shared)
Boot=(Type=Boot, Filename="%GAMEDIR%DerivedDataCache/Boot.ddc", MaxCacheSize=512)
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=-1, Path=%ENGINEDIR%DerivedDataCache, EnvPathOverride=UE-LocalDataCachePath)
Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=false, UnusedFileAge=8, FoldersToClean=-1, Path=?EpicDDC, EnvPathOverride=UE-SharedDataCachePath)
AltShared=(Type=FileSystem, ReadOnly=true, Clean=false, Flush=false, DeleteUnused=false, UnusedFileAge=8, FoldersToClean=-1, Path=?EpicDDC2, EnvPathOverride=UE-SharedDataCachePath2)
PakWrite=(Type=WritePak, Filename=../../../Enterprise/DerivedDataCache/Compressed.ddp, Compressed=true)

[CreateProjectCache]
MinimumDaysToKeepFile=7
Root=(Type=KeyLength, Length=120, Inner=AsyncPut)
AsyncPut=(Type=AsyncPut, Inner=Hierarchy)
Hierarchy=(Type=Hierarchical, Inner=Local, Inner=Project, Inner=Shared)
Local=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=-1, Path=%ENGINEDIR%DerivedDataCache, EnvPathOverride=UE-LocalDataCachePath)
Project=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=true, PurgeTransient=true, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=-1, Path=%GAMEDIR%ProjectDerivedData)
Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=false, UnusedFileAge=8, FoldersToClean=-1, Path=?EpicDDC, EnvPathOverride=UE-SharedDataCachePath)
AltShared=(Type=FileSystem, ReadOnly=true, Clean=false, Flush=false, DeleteUnused=false, UnusedFileAge=8, FoldersToClean=-1, Path=?EpicDDC2, EnvPathOverride=UE-SharedDataCachePath2)

[VirtualTextureChunkDDCCache]
UnusedFileAge=10
MaxFileChecksPerSec=-1 ;no limit
Path=%GAMEDIR%DerivedDataCache/VT

[DDCCleanup]
TimeToWaitAfterInit=120
TimeBetweenDeleteingDirectories=5
TimeBetweenDeletingFiles=2

 

注1:可在 DefaultEngine.ini 中创建新的[YourDDCSettings]条目的DDC配置,然后使用 -ddc=YourDDCSettings 运行编辑器来使用自定义DDC配置。

注2:也可以使用DefaultEngine.ini 中已定义好的DDC配置,如[NoShared],使用-ddc=NoShared来运行编辑器

 

DDC日志

[2023.09.22-10.43.59:750][     0]LogDerivedDataCache: Display: |DerivedDataBackends.cpp:774|Max Cache Size: 512 MB
[2023.09.22-10.43.59:808][     0]LogDerivedDataCache: |MemoryDerivedDataBackend.cpp:306|Loaded boot cache 0.06s 91MB ../../../../MyGame/DerivedDataCache/Boot.ddc.
[2023.09.22-10.43.59:808][     0]LogDerivedDataCache: Display: |DerivedDataBackends.cpp:792|Loaded Boot cache: ../../../../MyGame/DerivedDataCache/Boot.ddc
[2023.09.22-10.43.59:808][     0]LogDerivedDataCache: |DerivedDataBackends.cpp:288|FDerivedDataBackendGraph:  Pak pak cache file ../../../../MyGame/DerivedDataCache/DDC.ddp not found, will not use a pak cache.
[2023.09.22-10.43.59:808][     0]LogDerivedDataCache: |DerivedDataBackends.cpp:459|Unable to find inner node Pak for hierarchical cache Hierarchy.
[2023.09.22-10.43.59:808][     0]LogDerivedDataCache: |DerivedDataBackends.cpp:288|FDerivedDataBackendGraph:  EnginePak pak cache file ../../../Engine/DerivedDataCache/DDC.ddp not found, will not use a pak cache.
[2023.09.22-10.43.59:809][     0]LogDerivedDataCache: |DerivedDataBackends.cpp:459|Unable to find inner node EnginePak for hierarchical cache Hierarchy.
[2023.09.22-10.43.59:829][     0]LogDerivedDataCache: |FileSystemDerivedDataBackend.cpp:404|Speed tests for ../../../Engine/DerivedDataCache took 0.02 seconds
[2023.09.22-10.43.59:829][     0]LogDerivedDataCache: Display: |FileSystemDerivedDataBackend.cpp:176|Performance to ../../../Engine/DerivedDataCache: Latency=0.02ms. RandomReadSpeed=236.07MBs, RandomWriteSpeed=59.80MBs. Assigned SpeedClass 'Local'
[2023.09.22-10.43.59:832][     0]LogDerivedDataCache: |DerivedDataBackends.cpp:605|Using Local data cache path ../../../Engine/DerivedDataCache: Writable
[2023.09.22-10.43.59:832][     0]LogDerivedDataCache: |DerivedDataBackends.cpp:569|Shared data cache path not found in *engine.ini, will not use an Shared cache.
[2023.09.22-10.43.59:833][     0]LogDerivedDataCache: |DerivedDataBackends.cpp:459|Unable to find inner node Shared for hierarchical cache Hierarchy.

。。。

[2023.09.22-10.44.37:323][     0]LogDerivedDataCache: |MemoryDerivedDataBackend.cpp:197|Saved boot cache 0.05s 91MB ../../../../MyGame/DerivedDataCache/Boot.ddc.

注:通过加-logcmds="LogDerivedDataCache Verbose"参数来运行编辑器,可以打印更详细的DDC日志

 

参考

派生数据缓存(DerivedDataCache)

简单了解Unreal Engine中的DDC(Derived Data Cache)

 

posted on 2023-09-23 16:21  可可西  阅读(2355)  评论(1编辑  收藏  举报

导航