Search

Co de Ha rm on ics

mastering the routine play

Category

Technical Writing

Build a solution 

Start with a solution. Show the UI. Describe the components. And ask yourself – How did I get here? What have I done?!

(David Byrne)

What is a search experience?

An Algolia search experience is what an end user sees on the screen. Analyzing the end user experience helps you understand what we want to build, but it doesn’t help you understand the work that goes into building it.

Behind every search experience is another kind of experience: the Developer eXperience (DX) and the Business User eXperience (BuX). By providing great DX and BuX, Algolia ensures sure that its customers have it easy when developing a search experience. 

This document is about building a search experience. We organize it in the following way: we trace the main steps that all customers must follow to build a search solution.

Web Development

In the real world, there is no standard workflow for developing a website. Some companies will build the UI before plugging Algolia into its finished website. Others will start with their product data – seeing how to search and make sure that results are good, before building the search UI around that. Still others will do these in parallel. 

What matters is that all the necessary steps are done at some point in time. There are 4 steps to build a solution:

  • Data: Formatting and sending product data to Algolia, creating an index.
  • Configuration: Optimizing the data and engine-level settings.
  • UI: Building a vibrant front-end search experience.
  • Analytics: Tracking user behavior to continuously improve the search experience and ROI. 

We’ll begin with the data, then move to some essential configurations, then UI, then Analytics. We emphasize throughout that: this cycle is indeed a cycle that continues to roll – every step is revisited with feedback from the other steps, continuously evolving the solution to create the best experience. 

Data (Indexing)

The data is the customer’s data. Maybe this is obvious, but Algolia has no data. It has an empty space on its servers for customer data. The data that customers want their end users to search for must be located on Algolia’s servers. And for this, they must send Algolia its data in a process called Indexing

Indexing involves connecting to the server (via the API or Dashboard) and sending data. Once that’s done, the data is searchable. 

Customers can use the Dashboard to index their data, which is convenient for testing and troubleshooting, but definitely not the norm for sending large amounts of data. Live production data needs to be indexed by the API. The API allows developers to write a program that fetches their company’s data (located on their own databases and servers) and sends it to Algolia.

Let’s repeat that: Customer data needs to be fetched from the customer’s own databases. We say database but it could be a spreadsheet or an XML / CSV file or any other kind of source. This is what we call the Customer’s source data.

The kind of the data on the customer side is unimportant. What matters, during the Data stage, is the following:

  • Customers themselves have to create their own process (a “script”) that will find and collect their data on their computers;
  • Once the data is thus collected, they need to pick and choose the right information and reformat it into a new format called JSON (pronounced Jason). Jason is a human readable way to format data that Algolia accepts..
  • Once Jason is ready, it’s time for the customer to send (upload) it to Algolia. This is what we mean by Indexing.

So the first step is done – Algolia now has the customer’s data. The customer, at this point, should be able to test the search, to see if their upload worked. They can use either the Dashboard or the API. What they are testing is to see if Algolia has all the records they uploaded, and that all the information in their records is correct.

Configuration

Once that’s all done, the customer needs to keep thinking about its data, but in how they configure it. Configuration overlaps with Data. What do we mean by that? 

Configuration can be broken down into 3 parts: 

  • Data-centric configuration
  • Engine-level configuration
  • UI-centric configuration

Data-Centric Configuration

The first step for any client is to focus on the data-centric configuration, to make sure that the attributes they sent in their dataset are well chosen and used properly. Concretely: 

  • They need to select the attributes they want to use for searching: these need to be configured as Searchable Attributes. Title and Description are examples. 
  • Some are for ordering records: these need to be set up as Custom Ranking. Number of sales, or highest-rated are examples.
  • Some are for filtering: these need to be set up as Attributes for Faceting. Brand and color are examples.
  • The rest are for display (image) or for technical purposes (objectID). 

Actually, they overlap. Many attributes are good for both search and display (title, short description), and some are even good for search, display, and filtering (brand, color). 

Keep in mind, this implementation process is iterative. Customers will return to data and configurations, making small adjustments, as they move along in their implementation.

Build the UI

Once the data is set up properly, it’s time to start thinking about the UI. There are many examples of a great search UI. The best have a central search bar and an easy to read, vibrant set of results, with on-screen filters, pagination, and highlighting. Even better, a federated search with multiple kinds of data on the same screen (for example, Amazon’s books, films, appliances, and clothing showing up on the same results page, side-by-side, all matching the same query term).

Some of these options get us back to Data – How do we set up the data, for example, so that there are more filters? How about creating multiple kinds of data for the federated search? UI decisions also take us back to Configuration – the UI-centric ones, like pagination, and highlighting. Additionally, some configuration decisions are both data and UI – like on-screen faceting.

Once we’ve iterated back and forth between data, config, and UI, the website can be built. We offer developers out of the box tools (InstantSearch) to build a world-class search experience, whether it be on the web or mobile. With IS, developers need only focus on the design of the page, IS does all the work in connecting and communicating with the Algolia servers. Developers can also choose to use their own technologies, building everything from scratch. In either case, a solution is nothing without a great UI.

Once we have built the UI, and everything is set up, we’re ready to go live. Let’s make sure:

  • We have great and complete data; 
  • We paid careful attention to the essential configurations;
  • We tested our relevance and ranking; 
  • We built a vibrant UI for the end user. 

 

What’s left? Analytics

Analytics. You need to know what search does for your business. You want to find the relationship between Search and ROI. You want to quantify the business value that search brings. Only Analytics can help you do that. 

But Analytics has another purpose – one directly related to implementation. It’s the 4th step: Analytics brings us back to the Data <-> Config <-> UI iteration process. By tracking usage, over days, months, and years, we can analyze user behavior and continuously improve the experience, making sure that every query and click brings the most value. When looking at Analytics reports – for example, the statistics of “no results”, or “top searches”, or “most common misspellings” – and many more – all of this can help the customer reflect on their earlier decisions in the Data or Configuration steps. We can reformat data, add new tidbits, remove some data, and configure some of the more difficult “engine-level settings”, such as enable or disable typo tolerance, switch precision with attribute,  or turn on plurals and stop words.

That’s it. 

 

Exploring Technical Documentation Space

Awe. Like I am part of something insanely large. Touching infinity.

We travel in space. We are in space, and we and everything in it are moving.

To see this, you need to get out of the city and into the dark country. Lay back on the ground and face the sky.

It takes about 30 minutes for your eyes to adapt.

And then feel the movement.

Continue reading “Exploring Technical Documentation Space”

The iWalk Framework

Overview

In this documentation, we trace the multiple layers of the iWalk code, to demonstrate how data from the Crawl database gets to the iSee and iKnow web applications. The other direction – such as saving data entered by the Crawl user – is not discussed, though it follows the same layered process in reverse.

There are at least seven layers between the data in the Crawl database and the information that appears in iSee and iKnow. Seven is the lucky number, but there are in fact many ways to enumerate the quantity of layers. We can say there are actually ten or fifteen layers because within each layer there are a number of sub-layers of equal importance and significantly different from the others so as to warrant separate distinction. Or we can say there are really only two layers: the Data / Business layer and the Presentation Layer. But all this is too abstract. What follows is a discussion of the seven major interlocking points along the data cascade.

Sometimes the layers build upon the previous layer – combining fields, adding new data, and reformatting information; other times, the layers are redundant, with nothing really changing in the exchange, the only purpose being that of wrapping functionality or decoupling iWalk from the Crawl database. In all cases, however, the layers are set up to be followed in a very precise, linear fashion with the momentous task of getting data from Crawl (Part 1) into the front end (Part 2).

The first five layers – Part 1 – require a good understanding of Crawl’s data and its functional concepts. The final two layers – Part 2 – require a good understanding of iWalk functionality and web development.

Each layer plays a small but significant role in the overall process. Be aware that each layer accomplishes its task using a different way of thinking, a different vocabulary, a different technology, a different set of data, and a different way of representing that data.

This complexity requires patience, concentration, creativity and adaptability, along with an ease in a number of distinct skills – both functional and technical. And time – there is an estimated two-year learning curve. So, be patient with the process. At times, this document might let slip some criticisms of the iWalk Framework, or frustrations with its code. These are personal to the writer and are not intended to be merely critical: they are added constructively to inform the reader that the learning process is difficult for everybody.

Continue reading “The iWalk Framework”

Salesforce Crawl Platform

General Description

The Salesforce Interface (SFA) enables the exchange of Company data between Crawl and Salesforce, ensuring that both systems contain the same data. As of this writing, the systems that use SFA are Crawl, Salesforce, and iSee.

In the most common workflow, Salesforce is the Master of the data. This means that new company data is first entered into Salesforce and sent to Crawl using the automated SFA process. While the company remains a prospect, any changes to that data are done only in Salesforce and sent to Crawl. Crawl is therefore a Passive Receiver of data. Being passive, Crawl’s screens allow only read-only access to company data.

Note that there are other company data not in Salesforce that originate in Crawl and are therefore editable via Crawl.

All this changes when a company signs a contract with Sidewalk. With a signed contract, a prospect becomes a client, and the relationship switches, making Crawl the Master of the data and Salesforce the Passive Receiver. At this point, all changes to company data must be done in Crawl, which are automatically sent to Salesforce via the SFA interface. Additionally, Salesforce becomes read-only.

Continue reading “Salesforce Crawl Platform”

PRC Web Services

General Description

The PRC interface manages the exchange of data between Crawl and PRC. PRC is a third-party software and company used to perform credit checking on a company before contract signing.

The PRC process is a credit review process used by Crawl before contract signing.

The word “PRC” can be used in many different ways: the credit review process, or Crawl’s PRC interface, or the PRC software platform.

The “PRC software platform” stands for Platform Credit Risk, which is an in-house, Sidewalk solution to manage risk prior to contract signing. PRC is used by Sidewalk employees and interfaces, among others, with Nasaw and Crawl.

Continue reading “PRC Web Services”

Create a website or blog at WordPress.com

Up ↑