What do you like best about the product?

Great to collaborate with; very responsive; really appreciate their OHs to help out with issues that pop up; many out-of-the-box solutions for different kinds of ML models which really helped us out given the wide variety of ML models we run at the company.

What do you dislike about the product?

Nothing major to mention! We got everything resolved and the team was very helpful.

What problems is the product solving and how is that benefiting you?

Data Drift and ML Monitoring

What do you like best about the product?

One of the standout features of WhyLabs is its robust data observability capabilities. It provides continuous monitoring of data pipelines and ML models, allowing teams to quickly identify issues like data drift, model degradation, and training-serving skew. The platform's privacy-preserving integration ensures that data can be analyzed without moving or duplicating it, which is critical for maintaining security and privacy in sensitive industries like healthcare and finance​

What do you dislike about the product?

One potential drawback of WhyLabs is its relatively limited user reviews and feedback due to its newness in the market, making it harder for potential users to gauge its real-world performance at scale. This lack of detailed reviews can raise concerns about its maturity and support infrastructure​.Additionally, since it’s a newer platform, some advanced features might still be in development, and there could be steep learning curves for teams unfamiliar with observability tools in machine learning​.

What problems is the product solving and how is that benefiting you?

Data quality issues: It helps detect and address data drift and data integrity problems early, which is crucial for maintaining accurate and reliable ML models​

What do you like best about the product?

I like the privacy preserving solutions for scaling AI models. I like that WhyLabs offer responsive support and detailed documentation.

What do you dislike about the product?

I dislike that the platform might be overly technical for users who are not well-versed in AI or data science

What problems is the product solving and how is that benefiting you?

WhyLabs helps me solve issues like data drift and performance degradation in my AI models. This is crucial because I am working with sensitive medical data.

What do you like best about the product?

WhyLabs is the second observability platform I have ever used, and I can say the core features I like about the platform is that it is easy to set up and implement the features, the checks and metrics were already pre-loaded so I did not need to do much in configuring the application, and monitoring was not difficult to get started with. It also integrates well with the serving and data libraries we used for the production tutorial setup.

What do you dislike about the product?

Nothing so far, I only experienced a stability issues once (sometime in 2022), but support was able to help me quickly fix it.

What problems is the product solving and how is that benefiting you?

Since 2022, I have sparesely used WhyLabs to monitor the quality of datasets for one client and 2 customers (because it was not their core requirment but a nice piece of their stack to have).
whylogs seemed like the perfect choice for a consultant that clients did not want to entirely release their data to; I found that it only captures the profile and stats info instead of the raw data here.

Rcently, I started testing out LLM security features with LangKit and I cannot believe how quick it is to use. I followed a workshop few months ago that showed me how to detect jailbreak attempts and toxicity in LLM inputs and outputs using LangKit. Took that learning and now with a client's project, we have tested out logging the telemetary data from the evaluation to WhyLabs. Looks good so far, so once I upgrade the pricing limit for this client, we plan to scale our usage here. Excited about this one.

What do you like best about the product?

I've used WhyLabs for a few weeks and I was extremely pleased with it!
I will evaluate some dimensions of the tool that summarize my experience with it.

Easy Data Ingestion:
The ingestion API is straightforward to use and supports multiple connectors such as BigQuery, Databricks, and Spark, making data importation easy. Whylabs' use of Data Profiling ensures fast and secure data processing, eliminating the need to upload entire datasets, and making all the process very secure, since your data doesn't leave your servers.


Reliable Data Features:
Whylabs delivers all standard feature metrics accurately. Tracking data and model drift is very straightforward using Monitors.
Also, the platform supports custom metrics creation during or after ingestion.
Grouping by variables (segments) works well but must be defined during ingestion. Then you can analyze dataset features and track model performance per segment.


Flexible Monitors:
The monitoring system in Whylabs is highly adaptable and user-friendly, covering multiple variables with ease.
Monitors are easy to set up via the UI or JSON import, with summarized notifications for each monitor, keeping users informed without overwhelming them.
Additionally, monitors are JSON serializable, which is very helpful since you can track them with version control.


User-Friendly Usability:
Whylabs have a clean and intuitive UI, simplifying navigation for users.
While some advanced features may require programming knowledge, most tasks can be accomplished within the UI.
Thanks to data profiling, Whylabs delivers speedy performance without compromising on accuracy.


Solid Documentation:
The documentation provided by Whylabs is comprehensive and easy to understand, enabling users to make the most of the platform.


Pricing:
It's simply cheaper than its competition while having top notch features.


Customer Support:
They are always very helpful, answering all our questions and having several calls showcasing us different uses cases directly on the platform.


Overall, Whylabs offers a straightforward, efficient and affordable solution for monitoring Machine Learning models, with easy data ingestion, reliable feature analysis, and flexible monitoring options.

What do you dislike about the product?

There are a few cons:
- Dashboards are in beta, and while functional, they lack polish in terms of user interface. They are working actively on this, so probably a few months after this review this may be already fixed.
- Defining groupings by variables must be done at ingestion time, limiting flexibility for post-ingestion analysis.

That being said, they are very open to feedback and they may change or add features based on your needs.
In our case, dashboards were important and they are working on them.

What problems is the product solving and how is that benefiting you?

It solves most of the ML model monitoring needs that ML models often have while being affordable.