We can set the limit of no. of folders in browser tree by changing the value in properties file.
Open the Environment.properties file at \WEB-INF\classes\com\documentum\web\formext and change the value
DocbaseFolderTreeShowMoreNodeEnabled=true
DocbaseFolderTreeShowMoreThreshold=50
DocbaseFolderTreeShowMoreThreshold=50