sharepoint 2013 search operations
TRANSCRIPT
Robi Vončina• Newcomer dad• MVP, MCSE, MCT• Degree in economics• Started as MS Office instructor• System administrator• I started working with SharePoint from 2003 version on• Leader of SloSPUG
Agenda• Search components• Search workflow• Admin, crawl, content processing, query, index, analytics
• Search operations • Tools and scripts• Most common errors
• End User perspective• What are crawled and managed properties• What are display templates• Displaying custom search results with display templates
HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS
SharePointSP AppsDevicesNon-SP UX
SharePoint 2013 Search Workflow
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
Public APIUnit of scale/role boundary
Analytics Service
Feeding Chain Index Core Query Chain
HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS
SharePointSP AppsDevicesNon-SP UX
SharePoint 2013 Search Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
Public APIUnit of scale/role boundary
Feeding Chain
Crawl Component• OOB connectors• Extensible through
BCS• Local disk cache• Crawled items
tracked in Crawl database
• Configurations stored in Admin database
• Crawl modes• Full Crawl• Incremental Crawl• Continuous Crawl
Crawl
ContentProcessing Index
Crawl
HTTP
File Shares
SharePoint
User Profiles
Exchange
Lotus Notes
Documentum
Custom (BCS)
...Admin
mssearch.exe
SearchAdmin
FAST Search Index
Clarifying Guidance on SharePoint Security Groups versus Active Directory Domain Services Groups | Kirk Evans Bloghttps://blogs.msdn.microsoft.com/kaevans/2013/05/06/clarifying-guidance-on-sharepoint-security-groups-versus-active-directory-domain-services-groups/
Content Processing Component• Stateless node• Analyzes content for
indexing• Processing flow• Dictionaries• Schema mapping• Stores links and anchors in
Link database (analytics)• Extensible through web
service call-outs• Configurations stored in
admin database
Crawl
ContentProcessing Index
Crawl
Admin
AnalyticsProcessing
Link
Content EnrichmentWeb Service
SearchAdmin
FAST Search Index
HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS
SharePointSP AppsDevicesNon-SP UX
SharePoint 2013 Search Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
Public APIUnit of scale/role boundary
Index Core
Replica
ReplicaReplica
Index Partition detailsPartitions and Replicas• n Partitions• 1 Primary Replica• ≥ 0 Secondary ReplicasDetails
• Partition are a logical portion of the entire index
• Replicas are there for fault tolerance/ increased query throughput
• All nodes performs indexing• Journal shipping from primary to replicas• Partitions can be added but NOT
removed• Index can scale in both horizontal
(partitions) and vertical (replicas) ways
Replica
Content Processing
Replica
Partition #1
Replica
Replica
Replica
Partition #2
Replica
Replica
Replica
Partition #3
Replica
HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS
SharePointSP AppsDevicesNon-SP UX
SharePoint 2013 Search Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
Public APIUnit of scale/role boundary
Query Chain
SharePointSP AppsDevicesNon-SP UX
Web Front-End
UXQueryProcessing WFE
API
Index
• Query APIs• REST/OData API• Client-Side object model
(CSOM)• Server-Side object model
(SSOM)• Search center• Result templates• ContentWebPart• RefinerWebPart• Portals and publishing
Admin SearchAdmin
FAST Search Index
SharePointSP AppsDevicesNon-SP UX
Query Processing Component
UXQueryProcessing WFEIndex
• Stateless node• Processing flows• Query analyzer• Linguistics• Dictionaries• Result sources• Schema mapping• Query rules• Query federation • Configurations
stored in admin database
FAST Search Index
API
Admin SearchAdmin
HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS
SharePointSP AppsDevicesNon-SP UX
SharePoint 2013 Search Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
FAST Search Index
Public APIUnit of scale/role boundary
Analytics Service
Analytics Service empowers (examples)…• View counts• Sort by popularity• Recommendations• Relevancy based
on usage• Search reports• «Suggested sites
to follow»• «Are you looking
for these again?»
Analytics Processing Component
Crawl
ContentProcessing
QueryProcessing WFE
API
AnalyticsProcessing
LinkAnalytic
s Reportin
g
• Map-reduce• Learns by usage• Search Analytics• Usage Analytics• Enriches index by
updating index items
• Usage reports in Analytics database
Event Stream Admin
Index
FAST Search Index
SearchAdmin
Tools and scripts• Create Search Service application with PowerShell• X Blogs | Script for creating SharePoint 2013 Search service
applicationhttp://xblogs.kompas-xnet.si/post/Script-for-creating-SharePoint-2013-Search-service-applicationAdd or Modify SharePoint 2013 Search Topology using a PowerShell built User Interface - Scott Stewart - Site Home - TechNet Blogshttp://blogs.technet.com/b/scottstewart/archive/2015/05/21/add-or-modify-sharepoint-search-2013-topology-using-a-powershell-built-ui.aspx
Tools and scripts• Script Builds SP Search 2013 Index health reports for
monitoring and troubleshooting• https://gallery.technet.microsoft.com/scriptcenter/Builds-SP-Se
arch-2013-10d72a25• Get-SPSearchTopologyState script extended to check
the synchronization timer job• https://gallery.technet.microsoft.com/scriptcenter/Get-SPSearc
hTopologyState-b7452c6a
Case 1: Search component in „failed state“Suggested solution:• Restart server with
component in failed state
Case 2: New elements are not shown in search results• Open Central administration Service applications Search_SA Crawl log
Local SharePoint sitescrawl history Open first crawl log.
Case 2a: Search components OK no new elements are visible in search results• Error in crawl logs:»The crawler could not communicate with the server. Check that the server is available and that the firewall access is configured correctly. If the repository was temporarily unavailable, an incremental crawl will fix this error«
• Suggested solution:• Restart server or IIS on
crawl target server
Case 2b: Search components OK no new elements are visible in search results
• Error in crawl logs:»The item could not be indexed successfully because the item failed in the indexing subsystem. ( The item could not be indexed successfully because the item failed in the indexing subsystem.«
• Suggested solution:• Primary index component –
restart service or server
• Get-SPSearchTopologyState script extended to check the synchronization timer job• https://gallery.technet.micro
soft.com/scriptcenter/Get-SPSearchTopologyState-b7452c6a
Case 3: Opening crawl logs throws an errorOpen Central administration Service applications Search_SA Crawl log Local SharePoint sitescrawl history :• »Retrieving the COM class
factory for component with CLSID {0FF1CE15-0005-0000-0000-000000000000} failed due to the following error: 800703fa Illegal operation attempted on a registry key that has been marked for deletion. (Exception from HRESULT: 0x800703FA).«
• Suggested solution• Primary admin component
• Restart primary admin component
Degraded state:• After server restart• After service restart• Index component in degraded state
for too long
Case 4: Index component in »degraded state«• Suggested solution:
• Run Get-SPEnterpriseSearchStatus -Text -SearchApplication [search sa name]
• Run Get-SpEnterpriseSearchStatusExtended
• Usually all will show degraded state• Script Builds SP Search 2013 Index health
reports for monitoring and troubleshooting
https://gallery.technet.microsoft.com/scriptcenter/Builds-SP-Search-2013-10d72a25 • Look for:
• Index component [I.2.0] left behind• Not seeding!!!
• Restart search host controller service• Run script again
• Wait for sync to complete
Content Sources• Content Sources are the core to search
• Without content, what is the point of search?• Content can come from anywhere• Out of the box support includes
• SharePoint Sites• Web Sites• File Shares• Exchange Public Folders• Line of Business Data• Custom Repository
• External content with custom interfaces can be searched!• Utilize the BCS connectors to custom code a connector to a source
Types of Crawls• Full• Full crawls will re-index everything in a content source• Content, Columns, security ACLs
• Incremental• Only things that have changed (Added, modified, deleted)
• Continuous• Will continually crawl content that has changed• Does NOT crawl “instantaneously”
• Runs at 15 minute intervals• If not complete, will start new crawl thread
Crawl and Search Security• Crawl Security• By default, a single account is used to crawl all content
• Giving read access to all your content in the enterprise is frowned upon• Consider breaking up the Read account across the different
content sources to reduce exposure to hackers• BCS Connectors can be used to get security ACLS from custom
sources• Search Security• Search results are security trimmed
• Only content a users can gain access to will show in the search results• Means that target must contain ACLs that search can index• Custom repositories must be built to map any internal
authentication scheme to Active Directory accounts
Columns, crawled and managed properties
Columns:
•CreatedBy•ModifiedBy
Crawled properti
es•ows_Created_x0020_By
•ows_Modified_x0020_By
Managed
property
• Author
Crawl property naming convention is:
ows_q_<Type>_<ColumnName>ows_r_<Type>_<ColumnName>ows_taxId_<Type>_<ColumnName>
Managed property naming convention is:
<ColumnName>OWS<Type>owstaxId<ColumnName>
Display templates• HTML files• Converted to js files• Two types• Search results web parts• Content by search web
parts
Control<property name="RenderTemplateId"
Group<property name="GroupTemplateId"
Item<property name="ItemTemplateId"
Item
Item