Researchers at ETH Zurich have designed the first programming language, Silq, that can be used to program quantum computers as simply, reliably and safely as classical computers. Silq is the very first high-level programming language for quantum computers. High-level programming languages are more expressive, meaning that they can describe even complex tasks and algorithms with less code.

Silq programming language

Silq is a new high level quantum programming language developed by ETH Zurich. The intent is do provide a language that enables code that is shorter and simpler, less error prone, and more intuitive that existing lower level programming languages. The developers indicate that this will provide programs with much fewer lines of code than other programming languages; on average -46% versus Q# and -38% versus Quipper. A key feature is that it provides for automatic uncomputation of temporary values. This is more complicated than resetting a temporary classical value because the temporary qubits are entangled with other qubits and resetting it by measuring it can have some unintended consequences.  For now, Silq does not generate code for any existing hardware backend and only supports its own simulator.

Silq is the first quantum programming language that is not designed primarily around the construction and functionality of the hardware, but on the mindset of the programmers when they want to solve a problem—without requiring them to understand every detail of the computer architecture and implementation.

Silq is also the first programming language that automatically identifies and erases values that are no longer needed. It’s a major breakthrough in terms of optimizing the programming of quantum computers.

As a key novelty, Silq helps to bridge the conceptual gap between classical and quantum languages. As a consequence, Silq (i) lowers the entrance barrier for non-expert quantum programmers, (ii) generally helps to express complicated algorithms cleanly and concisely, and (iii) facilitates a tech-transfer of 50+ years of programming and analysis techniques developed for classical computing, to the domain of quantum programming.

Comparison

While traditionally quantum algorithms were often specified in terms of circuits, quantum languages express quantum algorithms more conveniently as source code. However, existing quantum languages force the programmer to work at a low level of abstraction, still essentially specifying quantum circuits that explicitly apply quantum operations to individual quantum bits. As a consequence, implementing quantum algorithms in these languages is tedious and error-prone.

In contrast, Silq supports a descriptive view on quantum algorithms that expresses the high-level intent of the programmer. Then, the compilation of these algorithms to low-level quantum circuits becomes a second-order concern that can be handled by a specialized compiler, much like in classical programming languages.

Experimental evaluation demonstrates that Silq programs are significantly shorter than equivalent programs in other quantum languages (on average -46% for ​Q#,​ -38% for ​Quipper​), while using only half the number of quantum primitives. Thus, Silq programs are not only shorter, but also easier to read and write, as they require less primitives and concepts.

Most of over evaluation focused on Q#, because (i) it is one of the most widely used quantum languages, (ii) we consider it to be more high-level than ​Cirq or ​QisKit​, and (iii) the Q# coding contests from ​2018 and ​2019 provide a large collection if Q# implementations we were able to leverage for our comparison.

Team​

Silq was developed by ​Benjamin Bichsel​, ​Maximilian Baader​, ​Timon Gehr​, and Prof. Martin Vechev​, who are all part of the SRI Lab, a research group at ETH Zurich’s Department of Computer Science.

Resources

• Silq’s webpage (​http://silq.ethz.ch/​) offers a detailed introduction of Silq’s ​key concepts ​, a demonstration of ​Silq’s benefits over Q# ​, concrete ​examples of Silq algorithms, instructions​ on installation, and much more.
• Silq’s parser, type checker, and simulator are publicly available at GitHub and licensed under the Boost Software License 1.0.

Key Benefit – Automatic Uncomputation

The key benefit of Silq is that it automatically (without intervention from the programmer) handles ​uncomputation of temporary values. In the following, we explain the necessity of uncomputation, and the benefits of automating it.

Analogously to the classical setting, quantum computations often produce temporary values. However, as a key challenge specific to quantum computation, removing such values from consideration induces an ​implicit measurement ​collapsing the state. In turn, collapsing can result in unintended side-effects on the state due to the phenomenon of ​entanglement​. Surprisingly, due to the quantum principle of ​deferred measurement​, preserving values until computation ends is equivalent to measuring them immediately after their last use, and hence cannot prevent this problem.

To remove temporary values from consideration without inducing an implicit measurement, algorithms in existing languages must explicitly uncompute all temporary values, i.e., modify their state to enable ignoring them without side-effects. In particular, even dropping the result of a sub-expression like ​x||y in ​(x||y)||z requires explicitly applying gates that uncompute this value, essentially forcing the programmer to perform manual memory management.

This results in a significant gap from quantum to classical languages, where discarding temporary values typically requires no action (except for heap values not garbage-collected). This gap is a major roadblock preventing the adoption of quantum languages as the implicit side-effects resulting from uncomputation mistakes, such as silently dropping temporary values, are highly unintuitive.

In contrast, Silq is comparable to high-level languages like Rust, in that it automatically handles uncomputation in a safe way. Among other benefits, this allows us to naturally support nested expressions, as known from classical languages.

Key Technique – Quantum Type System.​

Silq introduces a novel ​quantum type system which exploits a fundamental pattern in quantum algorithms, stating that uncomputation can be done safely if (i) the original evaluation of the uncomputed value can be described classically, and (ii) the variables used to evaluate it are preserved and can thus be leveraged for uncomputation.

Silq’s quantum type system allows it to detect this pattern, ensuring uncomputation is possible. Moreover, Silq also detects when uncomputation is not possible. In this case, Silq reports an error—before even running the algorithm. As a consequence, all Silq algorithms are physical: they can be realized on a quantum computer, and never attempt physically impossible operations like invalid uncompuation or reversing a measurement.

Future Benefits.

Silq is expected to advance various tools central to programming quantum computers. First, Silq compilers may require fewer quantum bits, e.g., by more flexibly picking the time of uncomputation. Second, static analyzers for Silq can safely assume (instead of explicitly proving) temporary values are discarded safely. Third, Silq simulators may improve performance by dropping temporary values instead of explicitly simulating uncomputation. Finally, Silq’s key language features are relevant beyond Silq, as they can be incorporated into existing languages such as QWire or Q#.

The post Silq – First Intuitive Programming Language For Quantum Computers appeared first on Quantum: Machine Learning & Analytics.

ProjectQ is an open-source software framework for quantum computing implemented in Python. It allows users to implement their quantum programs in Python using a powerful and intuitive syntax. ProjectQ can then translate these programs to any type of back-end, be it a simulator run on a classical computer or an actual quantum chip including the IBM Quantum Experience platform.   Other hardware platforms will be supported in the future.

The main features of ProjectQ are

• a high level language to write quantum programs
• a modular and customizable compiler
• various hardware and software back-ends
• a library (FermiLib) to solve fermionic problems on a quantum computer

Compiler

The compilation framework consists of a main compiler engine, which executes a user-defined sequence of compilation steps by sending the circuit through a chain of so-called compiler engines. Each compiler engine manipulates the circuit to, e.g., reduce the number of gates or quantum bits required to run the quantum program. Engines further down the stack become more back-end-specific and take care of the mapping of the logical circuit to the layout of the back-end.

Back-end

After the circuit has passed through all compiler engines, it gets sent to the back-end. The back-end may be actual quantum hardware, a simulator run on a classical computer, a circuit drawing engine, etc.

Having all of these possibilities in one unified framework allows users to test their implementations using a simulator before running them on actual hardware or getting resource estimates for larger instances.

ProjectQ is an open-source software framework for quantum computing implemented in Python. It allows users to implement their quantum programs in Python using a powerful and intuitive syntax.
Click To Tweet

The following back-ends are available in the first version of ProjectQ:

Simulator –  Simulate your quantum programs locally with ProjectQ’s high-performance simulator.

Emulator– Check the correctness of your quantum algorithm by running it on our emulator which employs classical shortcuts wherever it can

IBM Quantum Experience – Run your quantum program directly on IBM’s 5-qubit quantum computer.

Resource Counter – Determine gate counts and the maximal width of your circuit.

Drawing Engine – Draw your quantum program as a customizable LaTeX circuit for publication.

Command Printer – Check the stream of commands which gets sent to the back-end or save a low-level representation of your quantum program.

CirqProjectQ

CirqProject Q is a port between ProjectQ and Cirq that provides two main functions.  First, it is a ProjectQ backend that converts a ProjectQ algorithm to a cirq.Circuit.  And second, it can decompose ProjectQ common gates to native Xmon gates that can be used to simulate a Google quantum computer with ProjectQ.

ProjectQ performance

ProjectQ has a high-performance simulator which allows simulating up to about 30 qubits on a regular laptop.  Using the emulation features of our simulator (fast classical shortcuts), one can easily emulate Shor’s algorithm for problem sizes for which a quantum computer would require above 50 qubits, see our example codes.

The advanced features of the simulator are also particularly useful to investigate algorithms for the simulation of quantum systems. For example, the simulator can evolve a quantum system in time (without Trotter errors) and it gives direct access to expectation values of Hamiltonians leading to extremely fast simulations of VQE type algorithms.

The post ProjectQ appeared first on Quantum: Machine Learning & Analytics.

Honeywell’s team of scientists, engineers and technicians has built the highest performing quantum computer available.

With a quantum volume of 64, the Honeywell quantum computer is twice as powerful as the next alternative in the industry. That means that the company is closer to industries leveraging our solutions to solve computational problems that are impractical to solve with traditional computers.

With a quantum volume of 64, the Honeywell quantum computer is twice as powerful as the next alternative in the industry.
Click To Tweet

What makes Honeywell’s quantum computers so powerful is having the highest quality qubits, with the lowest error rates.  This is a combination of using identical, fully connected qubits and precision control.

What does a quantum computer look like?

First, there is an ultra-high vacuum chamber. It’s a stainless steel sphere, about the size of a basketball, with portals to allow in laser light, and from which the air has been pumped out such that it contains a vacuum of five times less particles than outer space. The chamber is cryogenically cooled with liquid helium to bring the temperature of the ion trap chip to 10 degrees above absolute zero (approximately negative 441 degrees Fahrenheit, colder than the surface temperature of Pluto).

Within the chamber, electric fields levitate individual atoms 0.1 mm above an ion trap, a silicon chip covered in gold about the size of a quarter. Scientists shine lasers at these positively charged atoms to perform quantum operations.

Meanwhile, there is a lot of equipment to control the quantum computer. Control systems are necessary to precisely manipulate the hundreds of independent electrical signals required to move the ions (qubits) in the intricate dance used for quantum information algorithms. Since the operations are all done with lasers, a multitude of optics are aligned across the optical tables, each specified for the correct color of light. All of this infrastructure takes up roughly two large optical tables (approximately 5 feet wide and 20 feet long), a significant space since the true computing power is harnessed with the few atoms hovering over the trap surface.

What will a quantum computer be used to solve?

Quantum computers harness their power by being able to investigate many potential outcomes at the same time.

Traditional computing bits are in a state of either “0” or “1.” On the other hand, Quantum bits, called qubits, can be in both states at the same time, a property called superposition.

That means that when you have these qubits interacting with each other in a computation, you get, what I call, a quantum superpower. There is an exponential expansion in the number of values that can all be considered at the same time.

Practically, that means that certain computations that can never be done on even the highest performing supercomputer will one day be computed on a quantum computer. One example is the best path of robots in a distribution center to improve speed of selecting items and packing orders (the best answer even supercomputers can’t find!).

Those computations are processed through algorithms designed and developed specifically for quantum computers. These algorithms are similar to classical algorithms from computer science, but also leverage a combination of physics and mathematics expertise. Within quantum’s ecosystem are a number of quantum-algorithm experts, who specialize in “converting” real world problems into quantum-standard algorithms. Honeywell Ventures has invested and partnered with two of the companies that do that, Zapata Computing and Cambridge Quantum Computing.

Future of quantum computing

When Honeywell set out to build a quantum computer, it prioritized creating the highest quality qubits, focusing on eliminating the errors present within the system on smaller numbers of qubits and then working to scale up the number of qubits.

Within quantum’s ecosystem are a number of quantum-algorithm experts, who specialize in “converting” real world problems into quantum-standard algorithms. Honeywell Ventures has invested and partnered with two of the companies that do…
Click To Tweet

By having achieved such low errors on quantum operations, with each new qubit it add to the machine, the quantum volume expands. That increases the capability of quantum computing that we will be able to bring to our customers. And thanks to a partnership with Microsoft’s Azure Quantum, it will be able to offer organizations access to its quantum computer both directly through its interface as well as through the Azure Quantum portal.

Scientists and businesspeople hope quantum computers will one day be able to solve seemingly intractable mathematical problems. These range from enhanced modeling of chemical reactions, which could help people discover new drugs or develop more environmentally friendly fuels, to making machine learning techniques more efficient, among other benefits. Honeywell expects to boost its quantum computing power by a factor of 10 each year for the next five years.

When Honeywell set out to build a quantum computer, it prioritized creating the highest quality qubits, focusing on eliminating the errors present within the system on smaller numbers of qubits and then working to scale up the number of…
Click To Tweet

About Honeywell

Honeywell International Inc. is an American company that is a publicly traded, multinational conglomerate. Honeywell is now headquartered in Charlotte, North Carolina, United States composed of four Business Groups—Aerospace, Building Technologies, Performance Materials & Technologies (PMT), and Safety & Productivity Solutions (SPS)—which produce aerospace systems, industrial products, and engineering services. Honeywell is a Fortune 100 company, and was ranked 77th in 2018. The company has a global workforce of approximately 110,000 workers with approximately 44,000 employed in the United States.

The post Honeywell Announced World’s Highest Performing Quantum Computer appeared first on Quantum: Machine Learning & Analytics.

Chinese internet giant Baidu Inc. has unveiled a new toolkit for quantum machine learning, known as Paddle Quantum. The toolkit, newly open-sourced Wednesday, comes with several quantum computing applications and is meant to be used by developers to build and train quantum neural network models. It’s built atop the company’s deep learning platform PaddlePaddle, which is used power its artificial intelligence services.

Paddle Quantum can be used to support quantum circuit models and general quantum computing research. For example, it can be used to simplify the implementation of a promising quantum algorithm, called the Quantum Approximate Optimization Algorithm, by 50%.Researchers in the quantum field can use the Paddle Quantum to develop quantum artificial intelligence.

Chinese internet giant Baidu Inc. has unveiled a new toolkit for quantum machine learning, known as Paddle Quantum.
Click To Tweet

Baidu also announced a number of new tools that provide a total of 27 enhanced features for the PaddlePaddle framework. The tools include Paddle.js, which is a deep learning JavaScript library that enables developers to use AI within a browser or mini-programs for applications such as WeChat.

Paddle Quantum Features

Paddle Quantum, available now on GitHub, consists of a set of quantum machine learning toolkits, including quantum development tools, a quantum chemistry library, and a set of optimization tools. It also supports three quantum applications: quantum machine learning, quantum chemical simulation, and quantum combinatorial optimization.

Paddle Quantum’s ease-of-use feature enables developers to build a simple quantum neural network or create their models following step-by-step quantum machine learning cases.

Developers also stand to benefit from Paddle Quantum’s versatility and scalability. Baidu has systematically added a number of underlying functions to PaddlePaddle, including complex variable definitions and matrix multiplications that allow Paddle Quantum to support quantum circuit models, as well as general quantum computing research.

Paddle Quantum’s ease-of-use feature enables developers to build a simple quantum neural network or create their models following step-by-step quantum machine learning cases.
Click To Tweet

Many quantum computing applications focus on solving combinatorial optimization problems and quantum chemistry simulations. Paddle Quantum provides a set of tools dedicated to these critical problems that our developers can experiment with directly. Baidu has also introduced some new quantum machine learning models that lay a foundation for applications like Gibbs state preparation, an essential task for quantum computation.

Along with Paddle Quantum, Baidu disclosed a novel implementation of the Quantum Approximate Optimization Algorithm (QAOA), which was proposed in 2014 to solve NP-hard Maximum cut problem. The Max-cut problem consists in finding a subset S of a graph’s vertices such that the number of edges between S and the complementary subset is as large as possible. This problem has applications in theoretical physics, VLSI circuit design, and other fields.

Another tool is Parakeet, which is a text-to-speech toolkit that supports models including ClariNet and WaveFlow. Then there’s Paddle X, which is an integration development tool that enables faster data processing.

Quantum machine learning prospects

Baidu’s newest toolkit is a sign of maturation of the quantum machine learning market, where the battle has moved away from hardware architecture to software, platforms and frameworks. As with all open source projects, adoption matter and we have to see how widely Paddle is being used in a few months.”

Baidu also provided a general update on PaddlePaddle’s adoption, saying it’s now being used by more than 1.9 million developers at 84,000 businesses. So far, about 230,000 deep learning models have been created using the platform.PaddlePaddle has unprecedented opportunity for the development, given the rise of industrial intelligence and the acceleration of AI-powered infrastructure.

Baidu's Paddle Quantum is a sign of maturation of the quantum machine learning market, where the battle has moved away from hardware architecture to software, platforms and frameworks.
Click To Tweet

About Baidu

Baidu, Inc. is a Chinese multinational technology company specializing in Internet-related services and products and artificial intelligence (AI), headquartered in Beijing. Baidu was incorporated in January 2000 by Robin Li and Eric Xu.

Baidu offers various services, including a Chinese search engine, as well as a mapping service called Baidu Maps. Baidu offers about 57 search and community services, such as Baidu Baike (an online encyclopedia), Baidu Wangpan (a cloud storage service), and Baidu Tieba (a keyword-based discussion forum).

Here you can read more about Tensorflow Quantum and Azur Quantum

The post Baidu Paddle Quantum appeared first on Quantum: Machine Learning & Analytics.

Dataiku DSS (Data Science Studio) is a collaborative data science platform designed to help scientists, analysts, and engineers explore, prototype, build, and deliver their own data products with maximum efficiency. DSS can run locally, within a database or in a distributed environment. It also integrates with Python and R. DSS allows the user to push computations to different engines (e.g., R, Python, Database, Hadoop/Spark).

One of DSS’s main strengths lies in its collaboration features. The software offers the possibility to work jointly on modeling workflows, to add documentation to a workflow and to document changes via an integrated Git repository. In addition, it has useful data preparation functions that automate the process of feature engineering. Model deployment in DSS can be done using the automation node or the real time API. Recent developments have enhanced DSS’s reporting and visualization capabilities, user management and automated machine learning capabilities.

Dataiku DSS (Data Science Studio) is a collaborative data science platform designed to help scientists, analysts, and engineers explore, prototype, build, and deliver their own data products with maximum efficiency.
Click To Tweet

The multi-deployment software has an all-in-one analytics and data science system that includes integrated coding and visual interface. This allows the use of notebooks such as R, Python, Hive, Spark, and more. A customizable drag and drop visual interface may also be used at any part of the predictive dataflow prototyping process – from wrangling, analysis, and modeling.

Data agnostic integration with over 30 data connectors and custom plugins extensions connect users to existing infrastructure. Dataiku DSS also detects data format and schema, and allows push computation to existing SQL, Hadoop, or Spark Infrastructure.

A visual profile of organizational data at each portion of an analysis allows users to interactively explore, prepare, enrich, blend, and clean data. The technology also leverages on Machine Learning such as Scikit-Learn, MLlib, XGboost, and more.

Dataiku DSS has a visual UI and tools to create and upgrade models in Python or R, for instant visual and statistical feedback on model performance. Production Deployment solutions bundle an entire workflow, while Monitoring and Version Control ensures that deployments run with proper data validation policies.

Dataiku DSS has a visual UI and tools to create and upgrade models in Python or R, for instant visual and statistical feedback on model performance.
Click To Tweet

Dataiku is an innovative software for those who develop using open source languages and need to add platform capabilities such as deployment, collaboration and user management. It offers interactive dashboards with a variety of charts, including a map engine for spatial data, as well as support for Web Apps, which makes it possible to customize and display any kind of visualization. The software is available on-premises and for cloud deployment and is accessed via a web browser

Collaboration

Integrated documentation and knowledge sharing: Dataiku is designed from the ground-up for data teams. Collaboration features make it easy to share knowledge amongst team members and onboard new users much faster.You can add detailed descriptions on your Dataiku objects (datasets, code, models…), tag, comment and favorite any Dataiku objects. Dataiku DSS also allows to engage with other users of the platform through Discussions and create Wikis to document your projects

Change management: Every action in the system is versioned and logged through an integrated Git repository. You can follow each action in the timeline in the interface and also easily rollback to previous versions.

Team activity monitoring: Dedicated dashboards help project managers keep an eye on their team’s activity. There are active and inactive projects and you can monitor team’s progress and commits.

Model Deployment

Easy to deploy: Software empowers analysts and data scientists to deploy models into production in a few clicks.Data cleaning, enriching, preprocessing, as well as models, are bundled together for simplified scoring pipelines.Deployed models are versioned, enabling users to deploy new versions, compare them and rollback at anytime.

Scalability & high availability:  You can handle large quantities of real-time predictions with queuing, parallelism, and load balancing and run multiple scoring nodes for full high availability. There is also automatic elastic scaling to handle unexpected traffic surges.

Deployment with Kubernetes:  You can deploy your API on-premises or in the cloud with fully native integration of Kubernetes for elastic and reproducible deployments. Full GPU support for deep-learning models is also available.

Powerful API engine: Deploy as an API visual models, custom Python or R models, custom Python or R functions or SQL queries. REST API is easy to use. There is also automatic generation of ready-to-use code samples.

Feedback loop: You can run multiple versions of the same model at the same time for automated A/B testing and monitor data changes over time. Access history of logs queries and predictions at any time to check that model performance is not drifting with time.

Visual Machine Learning and Modeling

Automated machine learning: Automatic feature engineering, generation, and selection to use any kind of data in your models. Consuct optimization of your model hyperparameters using various cross validation strategies and compare dozens of algorithms from Dataiku interface, both for supervised and unsupervised tasks. There is also available instant visual insights from your model (variables importance, features interactions or parameters), and assess model’s performance through detailed metrics.

Deep learning: With Dataiku DSS you can define your model architecture and personalize training settings. There is support for Keras, Tensorflow backend, integrate with Tensorboard, and scale model training using GPU. It also allows to automatically handle images, including features extractios and use pretrained models right from Dataiku interface.

Machine learning in production: As soon as your model is built and assessed, use it for batch scoring within your data workflow and deploy it as a real-time prediction service (REST API). Dataiku DSS give use all instrument to manage easily your model lifecycle: deploy new versions, retrain previous versions and rollback to any secure version in just one click and also to control your model’s performance over time with a feedback loop.

Dataiku DSS Benefits

With Dataiku DSS, converting raw data into useable, real-time predictions requires only one interface from start to finish. Users can explore, wrangle, and prepare without worrying about issues regarding format, storage, accesibility, and the like. The platform boasts of more than 25 connectors, with the option to create your own. Users can access all kinds of data anytime – from big or small, structured or unstructured, internal or external.

Dataiku’s Quick Columns View allows users to see the quality of data in real-time – from duplicates and invalids, to completeness and accuracy, to distribution and outliers. Full statistical summaries are also provided with just one click.

Dataiku DSS has data preparation tools for fast and reliable advanced analytics and predictive modeling.  A spreadsheet type interface, plus automatically suggested contextual transformations, makes performing mass actions on data less tedious for everyone.

Dataiku’s Quick Columns View allows users to see the quality of data in real-time – from duplicates and invalids, to completeness and accuracy, to distribution and outliers. Full statistical summaries are also provided with just one…
Click To Tweet

Retrieval is not a problem as Dataiku DSS as more than 90 built-in visual processors helps filter, search, and run code-free wrangling. Users may also customize solutions with business-specific and tailored transformation types. Formulas and Python scripts also allows custom processing.

Dataiku DSS Dashboards make it easy to create interactive charts and visualizations from datasets. 25 built-in chart formats – such ashistograms, to boxplots, to maps – allows drag-and-drop of data to automatically compute charts on existing Big Data infrastructure (SQL or Impala), thus resulting to optimal performance.

About Dataiku

Dataiku is a computer software company headquartered in New York City. The company develops collaborative data science software marketed for big data.The company was founded in Paris in 2013 by 4 co-founders. Two of them met while working at French search engine company Exalead, including chief executive Florian Douetteau, and Clément Sténac.

Dataiku opened an office in New York City in 2015 which became the company headquarters.They opened an office in London in the summer of 2016, and announced an office in Sydney in February 2019. The software Dataiku Data Science Studio (DSS) was announced in 2014, supporting predictive modelling to build business applications.Later versions of DSS added other features. Dataiku offers a free edition and enterprise versions   vcwith additional features, such as multi-user collaboration or real-time scoring.

Join our Facebook group to get the latest updates

The post Dataiku DSS – Data Science Studio appeared first on Quantum: Machine Learning & Analytics.

dotData is the world’s first and only Full-Cycle Data Science Automation Platform. It helps you automated entire Enterprise AI and Machine Learning workflow. Unlike traditional AutoML platforms, dotData automates time-consuming data wrangling and feature engineering parts of AI process, accelerating  projects to days instead of months. Data ingestion & wrangling, automated feature engineering, AutoML, model operationalization are automated with dotData Enterprise.

dotData Enterprise was designed to provide point-and-click access to Automated Machine Learning for your data-savvy professionals. With support for complex table relationships and the ability to perform computations on billions of rows of data, dotData provides the scalability and performance you need to create complex Enterprise AI models in record time.

dotData begins by providing automation features that help you connect to and prepare your data. dotData connects to your data and generates the required schemas to interpret table relationships. dotData goes beyond traditional AutoML features like raw data cleansing, and data partitioning with unique entity relationship extraction and automated table joins, aggregations and groupings to accelerate feature engineering.

dotData is the world’s first and only Full-Cycle Data Science Automation Platform. It helps you automated entire Enterprise AI and Machine Learning workflow. Data ingestion & wrangling, automated feature engineering, AutoML, model…
Click To Tweet

dotData automatically generates features from different types of source business data with complex relations. With support for transactional, text, geo-location, temporal as well time-series data, dotData’s automated Feature engineering analyzes millions of possible features and exposes the ones that are most likely to provide value to your organization, saving your company time and giving you the ability to uncover hidden insights.

dotData supports all state of the art Machine Learning libraries and provides Machine Learning optimization features like an automated hyper-parameter search of ML algorithms, the ability to select promising features for machine learning and the automatic selection of “champion” models.

dotData goes beyond just automation with support for a wide variety of state of the art machine learning libraries like scikit-learn, XGBoost, LightGBM, TensorFlow and PyTorch.

All dotData models can be deployed immediately using an innovative API approach. Create production-ready data and feature pipelines via APIs and create a production-ready ML scoring pipeline. It is possible to validate model accuracy automatically and retrain models based on real-world data using API-based integration. dotData makes deploying and maintaining AI models fast, simple, and cost-effective, giving you the best ROI and the fastest time to market possible.

dotData AI-FastStart Program

otData’s AI-FastStart program is an all-inclusive package of AutoML 2.0 Cloud-based software, professional services, ongoing training, and support all designed to work in concert to help Business Intelligence professionals accelerate their AI and Machine Learning development.

AutoML & Data Science Automation Library for Python

dotDataPy is a rich and scalable Python library that enables advanced users to access dotData’s data science automation functionality – including AI-powered feature engineering and automated machine learning – with Python code.

dotDataPy can be easily integrated with Jupyter notebooks and other Python development environments, enabling users to leverage the advanced Python ecosystem fully, including rich visualization like Matplotlib and Plotly, state-of-the-art machine learning/deep learning tools like scikit-learn, Spark MLlib, PyTorch, and TensorFlow, as well as flexible DataFrames like pandas and PySpark.

dotDataPy’s automated machine learning conducts hundreds of trials to finely tune state-of-the-art machine learning algorithms (including proprietary ones) for the best accuracy in various optimization criteria. The fully-automated process frees up the time and resources of data scientists and gives them the freedom to produce high-quality machine learning models, thus enabling teams to execute more data science projects than ever before.

dotDataPy can be easily integrated with Jupyter notebooks and other Python development environments, including Matplotlib and Plotly, scikit-learn, Spark MLlib, PyTorch, and TensorFlow, as well as flexible DataFrames like pandas and…
Click To Tweet

Deployment

The dotData platform supports on-premise Linux servers and a scalable Hadoop infrastructure. You can deploy dotData on your physical hardware to achieve best performance while maintaining strict controls over physical and logical access to ensure data protection, all while keeping your budget in check.

dotData’s engine can seamlessly scale to hundreds or thousands of servers to automatically build sophisticated machine learning models for your organization. Regardless of the size and breadth of your data set, dotData can seamlessly scale to match your needs. Manage the speed and efficiency of your data modeling capabilities seamlessly by adding or removing computing resources.

dotData’s platform provides enterprise-grade SSL protection for Hadoop cluster communication. AES encryption support allows stored user data to remain protected, while full HTTPS support means that communication between the dotData server and user browsers is fully encrypted. dotData also provides full Kerberos-based HDFS authentication support.

dotData is easily installed as a service on your Hadoop clusters and performs distributed automated feature generation and machine learning model scoring on HDFS. dotData leverages Spark for large-scale data processing and adheres to all Hadoop management policies.

dotData can be easily deployed on Amazon AWS to deliver the scale and full-cycle automation you need while providing scalability at a cost-effective rate. Deploy AutoML and Data Science Automation without committing to specific storage, computing or machine learning vendors.

he dotData platform supports on-premise Linux servers and a scalable Hadoop infrastructure as well as on Amazon AWS.
Click To Tweet

About dotData

dotData is focused on delivering end-to-end data science automation for the enterprise. dotData’s fully-automated data science platform speeds time to value by democratizing, operationalizing and accelerating the entire data science process, from source data through feature engineering to machine learning.

Unique to the dotData Platform is its AI-powered Feature Engineering, which eliminates the most time-consuming and labor- and skill-intensive aspects of the full data science process, freeing up data scientists for higher-value tasks and enabling data engineers and business analysts to drive data science projects.

dotData is a spin-off of NEC Corporation, and led by Dr. Ryohei Fujimaki, a world-renowned data scientist, and the youngest research fellow ever appointed in the 119-year history of NEC Corporation. The dotData team is delivering new levels of speed, scale and value in successful deployments across 10 industries, including Fortune Global 250 clients.

The post dotData AutoML 2.0 appeared first on Quantum: Machine Learning & Analytics.

Amazon SageMaker is a fully managed service that provides every developer and data scientist with the ability to build, train, and deploy machine learning (ML) models automatically. SageMaker enables developers to operate at a number of levels of abstraction when training and deploying machine learning models.

At its highest level of abstraction, SageMaker provides pre-trained ML models that can be deployed as-is. In addition, SageMaker provides a number of built-in ML algorithms that developers can train on their own data. Further, SageMaker provides managed instances of TensorFlow and Apache MXNet, where developers can create their own ML algorithms from scratch. Regardless of which level of abstraction is used, a developer can connect their SageMaker-enabled ML models to other AWS services, such as the Amazon DynamoDB database for structured data storage,AWS Batch for offline batch processing or Amazon Kinesis for real-time processing.

A number of interfaces are available for developers to interact with SageMaker. First, there is a web API that remotely controls a SageMaker server instance. While the web API is agnostic to the programming language used by the developer, Amazon provides SageMaker API bindings for a number of languages, including Python, JavaScript, Ruby, and Java. In addition, SageMaker provides managed Jupyter Notebook instances for interactively programming SageMaker and other applications.

Amazon SageMaker makes it easy to deploy your trained model into production with a single click so that you can start generating predictions for real-time or batch data. You can one-click deploy your model onto auto-scaling Amazon ML instances across multiple availability zones for high redundancy. Just specify the type of instance, and the maximum and minimum number desired, and SageMaker takes care of the rest. SageMaker will launch the instances, deploy your model, and set up the secure HTTPS endpoint for your application. Your application simply needs to include an API call to this endpoint to achieve low latency, high throughput inference. This architecture allows you to integrate your new models into your application in minutes because model changes no longer require application code changes.

Amazon SageMaker makes it easy to deploy your trained model into production with a single click so that you can start generating predictions for real-time or batch data.
Click To Tweet

Deploying with SageMaker

The development

Depending on where you are in your ML journey, or indeed your use case, you may choose to develop your own ML model rather than an off-the-shelf model. The next (also optional) step is to train the model. You can save any resulting model files to S3 for later user. You then need to write the inference code to build your API endpoint, that will serve the requests made to the model.

To speed up future development, as well to make it consistent and repeatable, you may choose to create a  ‘base’  Docker image with everything (but only what) you need for each project, and use that as the base for each new Dockerfile. Docker compose also comes in handy, especially for building and running locally to test before deploying.

Setting up the endpoint

When you’re ready to expose your model to the world, there will be three main steps to deploying your model with SageMaker: configuring the model, creating the endpoint configuration, and configuring the endpoint itself.

Create the model

As we configure the model in SageMaker, we provide the location of the model artefacts  and then the location of the inference code .This might seem like a lot of work, a little laborious and likely to be error prone.

The git commit triggers a CodeBuild project to start a job to package up the code as part of a Docker image. You could include the Dockerfile as part of the developer’s code repository, or fetch a standardized one from somewhere else.

The same CodeBuild project can even be used to make SageMaker SDK calls to configure the model in SageMaker, create the endpoint configuration and create the endpoint. You may also choose to specify where the model artefacts are located in S3 as part of the SageMaker configuration steps, instead of packaging them up in the Docker image.

Creating the endpoint configuration

This is where we start to do a bit more dynamic configuration – the production variants. This means firstly selecting the model that we just configured, you can even select multiple models as variants and apply weightings for A/B testing. Continue by selecting the instance type use, deciding how many instances to deploy, choose whether or not to enable elastic inference, and a few other small details.

Instance types

This is to the compute instance type to use, not dissimilar to launching a regular EC2 instance. Instance types are grouped into families based on core capabilities of the host computer hardware such as compute, memory, storage etc. Valid options can be found in the online documentation or the drop down list in the console. As of the current date, the available types include t2, m4, c4, c5, p2, and p3 instance classes each of varying capacity.

The p2 and p3 instances come under the Accelerated Computing family, launched with an attached GPU by default – useful for use cases such as deep learning.

Elastic inference

You may have some workloads that can benefit from the abilities of a GPU, but not require their use 24/7. Provisioning them anyway is financially inefficient, but luckily AWS introduced something called elastic inference (EI.) The EI provides acceleration when only required, and is automatically started when the supported frameworks are being used.

Latest supported frameworks can be found in the usual documentation but as of today some include EI-enabled versions of TensorFlow and MXNet. Although using a different deep learning framework, simply export your model by using ONNX, and then important your model into MXNet – use your model with EI as an MXNet model!

Configuring the endpoint

The last step in the process is to create an endpoint in SageMaker and apply the endpoint configuration you just created to it. You can reconfigure your endpoint later by creating new endpoint configuration and applying that to your endpoint instead.

Making it scale

SageMaker does do a lot of the heavy lifting for us already, but you may have noticed there are quite a few steps involved in the processes so far. To build a platform that enables users to autonomous and self-sufficient, we should probably abstract some of that away, so that they can focus on their domain knowledge.

Amazon SageMaker Studio 

Amazon SageMaker Studio provides a single, web-based visual interface where you can perform all ML development steps. SageMaker Studio gives you complete access, control, and visibility into each step required to build, train, and deploy models. You can quickly upload data, create new notebooks, train and tune models, move back and forth between steps to adjust experiments, compare results, and deploy models to production all in one place, making you much more productive. All ML development activities including notebooks, experiment management, automatic model creation, debugging, and model drift detection can be performed within the unified SageMaker Studio visual interface.

Amazon SageMaker Studio provides a single, web-based visual interface where you can perform all ML development steps. SageMaker Studio gives you complete access, control, and visibility into each step required to build, train, and…
Click To Tweet

Amazon SageMaker Autopilot

Amazon SageMaker Autopilot is the industry’s first automated machine learning capability that gives you complete control and visibility into your ML models. Typical approaches to automated machine learning do not give you the insights into the data used in creating the model or the logic that went into creating the model. As a result, even if the model is mediocre, there is no way to evolve it. Also, you don’t have the flexibility to make trade-offs such as sacrificing some accuracy for lower latency predictions since typical automated ML solutions provide only one model to choose from.

SageMaker Autopilot automatically inspects raw data, applies feature processors, picks the best set of algorithms, trains and tunes multiple models, tracks their performance, and then ranks the models based on performance, all with just a few clicks. The result is the best performing model that you can deploy at a fraction of the time normally required to train the model. You get full visibility into how the model was created and what’s in it and SageMaker Autopilot integrates with Amazon SageMaker Studio. You can explore up to 50 different models generated by SageMaker Autopilot inside SageMaker Studio so its easy to pick the best model for your use case. SageMaker Autopilot can be used by people without machine learning experience to easily produce a model or it can be used by experienced developers to quickly develop a baseline model on which teams can further iterate.

Amazon SageMaker Ground Truth

Successful machine learning models are built on the shoulders of large volumes of high-quality training data. But, the process to create the training data necessary to build these models is often expensive, complicated, and time-consuming. Amazon SageMaker Ground Truth helps you build and manage highly accurate training datasets quickly. Ground Truth offers easy access to labelers through Amazon Mechanical Turk and provides them with pre-built workflows and interfaces for common labeling tasks. You can also use your own labelers or use vendors recommended by Amazon through AWS Marketplace. Additionally, Ground Truth continuously learns from labels done by humans to make high quality, automatic annotations to significantly lower labeling costs.

Amazon SageMaker Experiments

Amazon SageMaker Experiments helps you organize and track iterations to machine learning models. Training an ML model typically entails many iterations to isolate and measure the impact of changing data sets, algorithm versions, and model parameters. You produce hundreds of artifacts such as models, training data, platform configurations, parameter settings, and training metrics during these iterations. Often cumbersome mechanisms like spreadsheets are used to track these experiments.

SageMaker Experiments helps you manage iterations by automatically capturing the input parameters, configurations, and results, and storing them as ‘experiments’. You can work within the visual interface of SageMaker Studio, where you can browse active experiments, search for previous experiments by their characteristics, review previous experiments with their results, and compare experiment results visually.

The ML training process is largely opaque and the time it takes to train a model can be long and difficult to optimize. As a result, it is often difficult to interpret and explain models. Amazon SageMaker Debugger makes the training process more transparent by automatically capturing real-time metrics during training such as training and validation, confusion matrices, and learning gradients to help improve model accuracy.

SageMaker Experiments helps you manage iterations by automatically capturing the input parameters, configurations, and results, and storing them as ‘experiments’.
Click To Tweet

SageMaker Debugger

The metrics from SageMaker Debugger can be visualized in SageMaker Studio for easy understanding. SageMaker Debugger can also generate warnings and remediation advice when common training problems are detected. With SageMaker Debugger, you can interpret how a model is working, representing an early step towards model explainability.

Amazon SageMaker Model Monitor

Amazon SageMaker Model Monitor allows developers to detect and remediate concept drift. Today, one of the big factors that can affect the accuracy of deployed models is if the data being used to generate predictions differs from data used to train the model. For example, changing economic conditions could drive new interest rates affecting home purchasing predictions. This is called concept drift, whereby the patterns the model uses to make predictions no longer apply. SageMaker Model Monitor automatically detects concept drift in deployed models and provides detailed alerts that help identify the source of the problem. All models trained in SageMaker automatically emit key metrics that can be collected and viewed in SageMaker Studio. From inside SageMaker Studio you can configure data to be collected, how to view it, and when to receive alerts.

Many machine learning applications require humans to review low confidence predictions to ensure the results are correct. But, building human review into the workflow can be time consuming and expensive involving complex processes. Amazon Augmented AI is a service that makes it easy to build the workflows required for human review of ML predictions. Augmented AI provides built-in human review workflows for common machine learning use cases. You can also create your own workflows for models built on Amazon SageMaker. With Augmented AI, you can allow human reviewers to step in when a model is unable to make high confidence predictions.

Amazon SageMaker Components

Amazon SageMaker Components for Kubeflow Pipelines, now in preview, are open-source plugins that allow you to use Kubeflow Pipelines to define your ML workflows and use SageMaker for the data labeling, training, and inference steps. Kubeflow Pipelines is an add-on to Kubeflow that let you build and deploy portable and scalable end-to-end ML pipelines. However, when using Kubeflow Pipelines, ML ops teams need to manage a Kubernetes cluster with CPU and GPU instances and keep its utilization high at all times to reduce operational costs. Maximizing the utilization of a cluster across data science teams is challenging and adds additional operational overhead to the ML ops teams.

As an alternative to an ML optimized Kubernetes cluster, with Amazon SageMaker Components for Kubeflow Pipelines you can take advantage of powerful SageMaker features such as data labeling, fully managed large-scale hyperparameter tuning and distributed training jobs, one-click secure and scalable model deployment, and cost-effective training through EC2 Spot instances without needing to configure and manage Kubernetes clusters specifically to run the machine learning jobs.

Kubernetes is an open source system used to automate the deployment, scaling, and management of containerized applications. Many customers want to use the fully managed capabilities of Amazon SageMaker for machine learning, but also want platform and infrastructure teams to continue using Kubernetes for orchestration and managing pipelines. SageMaker lets users train and deploy models in SageMaker using Kubernetes operators.

In most deep learning applications, making predictions using a trained model – a process called inference – can be a major factor in the compute costs of the application. A full GPU instance may be over-sized for model inference. In addition, it can be difficult to optimize the GPU, CPU, and memory needs of your deep-learning application. Amazon Elastic Inference solves these problems by allowing you to attach just the right amount of GPU-powered inference acceleration to any Amazon EC2 or Amazon SageMaker instance type or Amazon ECS task with no code changes. With Elastic Inference, you can choose the instance type that is best suited to the overall CPU and memory needs of your application, and then separately configure the amount of inference acceleration that you need to use resources efficiently and to reduce the cost of running inference.

Amazon SageMaker Python SDK

Amazon SageMaker Python SDK is an open source library for training and deploying machine-learned models on Amazon SageMaker.With the SDK, you can train and deploy models using popular deep learning frameworks, algorithms provided by Amazon, or your own algorithms built into SageMaker-compatible Docker images.

SageMaker Debugger provides a way to hook into the training process and emit debug artifacts (a.k.a. “tensors”) that represent the training state at each point in the training lifecycle. Debugger then stores the data in real time and uses rules that encapsulate logic to analyze tensors and react to anomalies. Debugger provides built-in rules and allows you to write custom rules for analysis.

Amazon SageMaker Application

Tens of thousands of customers utilize Amazon SageMaker to help accelerate their machine learning deployments, including ADP, AstraZeneca, Avis, Bayer, British Airways, Cerner, Convoy, Emirates NBD, Gallup, Georgia-Pacific, GoDaddy, Hearst, Intuit, LexisNexis, Los Angeles Clippers, NuData (a Mastercard Company), Panasonic Avionics, The Globe and Mail, and T-Mobile.

Since launch, AWS has regularly added new capabilities to Amazon SageMaker, with more than 50 new capabilities delivered in the last year alone, including Amazon SageMaker Ground Truth to build highly accurate annotated training datasets, SageMaker RL to help developers use a powerful training technique called reinforcement learning, and SageMaker Neo which gives developers the ability to train an algorithm once and deploy on any hardware. These capabilities have helped many more developers build custom machine learning models. But just as barriers to machine learning adoption have been removed by Amazon SageMaker, customers’ desire to utilize machine learning at scale has only increased.

About Amazon Web Services

Amazon Web Services is the world’s most comprehensive and broadly adopted cloud platform. AWS offers over 165 fully featured services for compute, storage, databases, networking, analytics, robotics, machine learning and artificial intelligence (AI), Internet of Things (IoT), mobile, security, hybrid, virtual and augmented reality (VR and AR), media, and application development, deployment, and management from 69 Availability Zones (AZs) within 22 geographic regions, with announced plans for 13 more Availability Zones and four more AWS Regions in Indonesia, Italy, South Africa, and Spain. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—use AWS to power their infrastructure, become more agile, and lower costs.

About Amazon

Amazon is guided by four principles: customer obsession rather than competitor focus, passion for invention, commitment to operational excellence, and long-term thinking. Customer reviews, 1-Click shopping, personalized recommendations, Prime, Fulfillment by Amazon, AWS, Kindle Direct Publishing, Kindle, Fire tablets, Fire TV, Amazon Echo, and Alexa are some of the products and services pioneered by Amazon.

Read more about automated machine learning

Join our Facebook Group

The post Amazon SageMaker appeared first on Quantum: Machine Learning & Analytics.

Business intelligence are systems that combine data gathering, data storage and knowledge management with analysis to evaluate complex corporate and competitive information for presentation to planners and decision maker, with the…
Click To Tweet

BI tools are types of software used to gather, process, analyze, and visualize large volumes of past, current, and future data in order to generate actionable business insights, create interactive reports, and simplify the decision-making processes.

These tools include key features such as data visualization, visual analytics, interactive dashboarding and KPI scorecards. Additionally, they enable users to utilize automated reporting and predictive analytics features based on self-service.The most efficient BI software are Power BI, Tableau, Seeq, ChartMogul, AktivTrak, Clear Analytics, Databox, Pipeline, Domo, Kyubit, MicroStrategy, SAP BusinessObjects, SAS Business Intelligence, Yellowfin BI, IBM Cognos Analytics, Cluvio, GoodData

The benefits of business intelligence software

Professional software and tools offer various prominent benefits, here we will focus on the most invaluable ones:

1. They bring together all relevant data: Whether you work in a small company or large enterprise, you probably collect data from various portals, ERPs, CRMs, flat files, databases, APIs, and much more. You need to obtain a high level of data intelligence to be able to manage all these sources and develop a better understanding of the collected information. That’s why utilizing modern data connectors will help you in centralizing the disparate sources and provide you with a single point of view on all your business processes. That way, identifying issues, trends, and taking action are closely connected and based solely on data.
2. Their true self-service analytics approaches unlock data access: When each person in the company is equipped with modern business intelligence software that will enable him/her to explore the data on their own, the need to request reports from the IT department is significantly reduced. This self-service BI approach gives organizations a competitive advantage because each employee will be equipped with the right amount of data analytics skills that will, ultimately, save the company’s time and resources while unburdening the IT department, hence, enabling them to focus on other critical tasks.
3. Users can take advantage of predictions: Predictive analytics doesn’t need to be a specialty of data scientists or analysts. With the integration of forecast engines business users can generate insights for future scenarios that will help them in adjusting current strategies to deliver the best possible results. On the other hand, if a business condition changes, intelligent data alerts safeguard the anomalies that can occur while you manage huge amounts of data, and discover new trends and patterns that will enable you to react immediately.
4. They eliminate manual tasks: While traditional means of business management encourage the use of spreadsheets and static presentations, modern software eliminates endless amounts of rows and columns and facilitates the automation of processes. The tool updates your KPI dashboard itself with real-time data. Besides, you can automate the reporting process with specified time intervals and purely look at the results. Simply drag-and-drop your values and see how you can easily create a powerful interactive dashboard that enables you to directly interact with your screen.
5. They reduce business costs: From sales planning and customer behavior analysis to real-time process monitoring and offer optimization, BI platforms enable faster planning, analysis, and reporting processes. In fact, according to a survey conducted by the BI survey, more than 50% of business intelligence users reported that these are the most prominent benefits that helped them reducing costs, and increasing revenues. If you can work fast and accurate, you can achieve far better business results and make profitable adjustments.
6. They’re constantly at your service, 24/7/365: Various organizations require various needs and the Software-as-a-Service model offered by these tools will provide a full SaaS BI experience, with all the data hosted in a secure online environment. According to your needs, the software can scale or de-scale, thus, adjusting to the specific needs of a company. Since the data is stored on a cloud, you have non-stop access to the software, where you can fully explore various self-service analytics features no matter if you’re a manager, data scientist, analyst or consultant.

These business intelligence benefits are focused both on small companies and large enterprises. If you need to control your data and know what’s going on in your company, BI is the way to do so. To help you on the way, we have created a business intelligence tools list that not only will ensure your data management and discovery become intuitive and easy to use, but also stay secured and guarantee a higher level of productivity, and, consequently, increased profits.,

Power BI

Power BI – business analytics software by Microsoft, which provides interactive visualizations and business intelligence capabilities with an interface simple enough for end users to create their own reports and dashboards.

Power BI made of 6 main components, these components released in the market separately, and they can be used even individually. Components of Power BI are:

• Power Query: Data mash up and transformation tool.
• Power Pivot: In-memory tabular data modelling tool
• Power View: Data visualization tool
• Power Map: 3D Geo-spatial data visualization tool
• Power Q&A: Natural language question and answering engine.
• Power BI Desktop: A powerful companion development tool for Power BI

There are many other parts for Power BI as well, such as;

• PowerBI.com Website; which Power BI data analysis can be shared through this website and hosted there as cloud service
• Power BI Mobile Apps; Power BI supported in Android, Apple, and Windows Phones.

Power BI components can be used individually or in a combination. Power Query has an add-in for Excel 2010 and Excel 2013, and it is embedded in Excel 2016. The add-in for Power Query is available for free! for everyone to download and use it alongside with existing an Excel (as long as it is Excel 2010 or higher versions). Power Pivot has been as an add-in for Excel 2010, from Excel 2013 Power Pivot is embedded in Excel, this add-in is again free to use! Power View is an add-in for Excel 2013, and it is free for use again. Power Map is an add-in for Excel 2013, it is embedded in Excel 2016 as 3D maps. Power Q&A doesn’t require any installation or add-in, it is just an engine for question and answering that works on top of models built in Power BI with other components.

Tableau

Tableau – an integrated business intelligence and analytics solution that helps to analyze key business data and generate meaningful insights. The solution helps businesses to collect data from multiple source points such as SQL databases, spreadsheets, cloud apps like Google Analytics and Salesforce to create a collective dataset.

Tableau’s live visual analytics and interactive dashboard allow slicing & dicing datasets for generating relevant insights and exploring new opportunities. Users can create interactive maps and analyze data across regions, territories, demographics and more. Tableau helps to create a narrative story of the data analysis with interactive visualizations that can be shared with their audience.

Tableau products include:

• Tableau Desktop (Both professional and personal editions)
• Tableau Server
• Tableau Online
• Tableau Prep Builder
• Tableau Vizable
• Tableau Public
• Tableau Reader

Tableau can be customized to serve a number of industry verticals including banking, communication, education, healthcare, real estate, manufacturing and technology. The solution can be either installed on-premises or hosted on the cloud as a SaaS application. Users can also view and analyze data on their Android and iOS smartphones with Tableau Mobile app.

Qlik Sense 

Qlik Sense  – a business and visual analytics platform that supports a range of analytic use cases. Built on Qlik’s unique Associative Engine, it supports a full range of users and use-cases across the life-cycle from data to insight

Qlik Sense sets the benchmark for third-generation analytics platforms, empowering everyone in organizations to make data-driven decisions.  People of all skill levels get unmatched analytical power, with unique associative exploration and search, AI-powered insight suggestions, AI-accelerated creation and data prep, advanced analytics integration, and natural-language, conversational interaction. They can easily share and communicate the insights they discover and get answers anywhere they work – with fully interactive and secure online and offline mobility. And for the enterprise, you get a true platform with open and standard APIs for customization and extension, powerful data integration and connectivity, centralized management and governance, and a Kubernetes based SaaS / multi-cloud architecture driving scalability across combinations of on-premise, private, and public cloud environments.

Features of Qlik Sense:

• Smart search options like Google and associative functions.
• Fast and reliable connections to multiple data sources.
• Drag and drop visualizations
• Generate personalized reports and detailed, interactive dashboards.
• Self-service data discovery

The Qlik Sense system offers data visualization and discovery for individuals and teams. The software’s data discovery tool helps businesses of all sizes explore simple and complex data and find all possible associations in their datasets. With the drag and drop interface, users can create interactive data visualizations to present the outcome in a story form.

Seeq

Seeq is an advanced analytical application fully integrated with the PI Data Archive and PI Asset Framework. Seeq delivers a hybrid approach to analytics, easily combining first principals, statistics, and machine learning. Seeq’s approach brings self-service analytics for engineers and scientists, collaboration and knowledge capture for teams, and publishing and real time views for managers and operators.

It enables users to get more value out of data, including OSIsoft PI System, Honeywell’s UniformancePHD, Emerson DeltaV and Ovation, Inductive Automation’s Ignition, AspenTech IP.21, Wonderware, GE Proficy or any other, by providing interactive, visual tools to accelerate industrial process analytics.

Seeq can be set up and running on a dedicated server, server cluster, or virtual machine in as little time as an hour. On-premise installations on the same network as a plant or enterprise historian, or on the cloud (Microsoft Azure, Amazon Web Services, etc.), or on a mixed environment of on-premise and cloud resources are supported.

This software is extensible through data export, data integration, and a REST API for creating custom templates and modules. Data export options include Microsoft Excel and PowerPoint, and any OData client (Tableau, Microsoft PowerBI, etc.). Data integration with OSIsoft Vision is supported, and the REST API has SDKs for programming in C#, Python, MatLab, and Java.

Seeq does not copy or duplicate data from the source of record. Instead, data is accessed via a connector retrieving data on the fly based on user actions. All Seeq documents such as workbooks, worksheets, topics, and search definitions are stored by Seeq for easy user access and sharing.

datapine

datapine is a BI software that lets you connect your data from various sources and analyze with advanced analytics features (including predictive). With your analysis, you can create a powerful business dashboard (or several), generate standard or customized reports or incorporate intelligent alerts to get notified of anomalies and targets. This tool, rated with outstanding 4.6 stars on Capterra, is a powerful solution for businesses of all sizes since datapine can be implemented for various industries, functions, and platforms, no matter the size.

The tool offers features for both advanced users such as data analysts and average business users. The SQL mode enables analysts to create their own queries while, on the other hand, the intuitive drag-and-drop interface ensures a visually intuitive way of entering your values and creating powerful charts and dashboards, simply by using effective visual analytics.

datapine is a BI software that lets you connect your data from various sources and analyze with advanced analytics features (including predictive). With your analysis, you can create a powerful business dashboard (or several), generate…
Click To Tweet

Key features:

1. Intuitive drag-and-drop interface: Simply drag your KPIs into the interface and see how your charts and graphs transform into a professional dashboard without any IT involvement or advanced coding.
2. Easy-to-use predictive analytics: Advanced analytics features, including predictive analysis based on a forecast engine, where you simply select the KPI, determine the number of data points you want to forecast, specify the previous data points for the calculation, and finally define the model quality by appointing the desired confidence interval.
3. Many interactive dashboard features: Dig deep into each part of the dashboard with advanced dashboard software features such as drill-downs, chart zooms, widget linking, hierarchical filters, among others, that will ensure a detailed data discovery process.
4. Multiple reporting options: Take advantage of BI reporting tools and create standard or customized reports by sharing via URL, enabling viewer access or sending an automated e-mail report which you can automate on specific time intervals. Additionally, you can embed datapine into your own application.
5. Smart insights and alarms based on artificial intelligence: They alert the user if a business anomaly occurs or generate multiple types of insights, including growth, trends, value drivers, what-if scenarios, among others.

They do offer mobile access, but the dashboards are not directly accessible unless you download their app and customize them within the mobile view.

SAS Business Intelligence

SAS Business Intelligence is a software solution offering numerous products and technologies for data scientists, text analysists, data engineers, forecasting analysts, econometricians, and optimization modelers, among others. Founded in the 70s, SAS Business Intelligence enjoys a long tradition in the market, building and expanding its products every year.

Key features:

• 5 main subjects: SAS offers different BI products and technologies that are focused on visual data exploration and easy analytics, interactive reporting and dashboards, collaboration, and mobile apps.
• Data exploration supported by machine learning: SAS’ data exploration offers automated analysis supported by machine learning compiled in interactive visualizations
• Text analytics capabilities: Interesting features are text analytics that enables you to gain insights from text data such as social media and location analytics that combines data in a geographical context.
• Reports and dashboards across devices: Dynamic visuals are organized into interactive reports and dashboards, similar to other tools in our lists, and you can access them also on mobile and tablet devices (both Android and iOS).
• Integration with other applications: You can integrate SAS with Microsoft Office applications such as PowerPoint, Word, SharePoint or Excel with their Office Analytics solution which enables you to view your results directly in Microsoft Office Suite.

Clear Analytics

Clear Analytics is a tool that consolidates data from internal systems, cloud, accounting, CRM, and allows you to drag-and-drop that data into Excel. It works with Microsoft Power BI, using Power Query and Power Pivot to clean and model different datasets. Capterra gives a high user review of 4.5 stars making this tool also one of the highest-rated on our list.

Key features:

• Reports delivered to Power BI: It delivers self-service ad hoc reporting directly to Power BI, hence enabling non-technical users to explore data stored in databases with their drag-and-drop interface and dynamic query designer.
• Connected with Excel: It bolts right into Excel, where you can specify end users’ viewing rights based on a user or role level, and manipulate formulas freely, whether you need to perform basic or advanced analysis.
• Sharing on mobile devices: You can use Power BI features such as Power Maps, Pivot and Desktop, where you can share your insights also to mobile devices, including iWatch.
• A full audit trail: They offer a full audit trail that enables you to know where the data came from, what filters were applied, who ran the report, and when the data was extracted.
• Fetch data elements with a semantic layer: A semantic layer will enable you to fetch data elements and organize them with modeling, visualizing, and transforming data to generate needed insights.

SAP BusinessObjects

SAP BusinessObjects is a business intelligence suite designed for comprehensive reporting, analysis, and data visualization. They provide Office integrations with Excel and PowerPoint where you can create live presentations and hybrid analytics that connects to their on-premise and cloud SAP systems. They’re focused on business categories such as CRM and customer experience, ERP and digital core, HR and people engagement, digital supply chain, and many more. To be accurate, more than 170M users leverage SAP across the world, making it one of the largest software suppliers in the world.

Key features:

• BI enterprise reporting system: Reporting and analysis are done with ad hoc queries and a complex BI enterprise reporting system designed to guide your overall company strategy.
• Self-service, role-based dashboards: Analytics applications and data visualization are focused on self-service capabilities, similar to other tools, where you can build your own role-based dashboards and applications to assess risks, for example.
• Cross-enterprise sharing: You can create a cross-enterprise sharing functionality that will enable you to communicate your findings with other stakeholders and partners.
• Connection with SAP Warehouse and HANA: Multidimensional data analysis enables you to filter and manipulate data with real-time analytics when you connect with SAP Business Information Warehouse (SAP BW) or SAP HANA.
• Integration with Office: By using Microsoft Excel with SAP, you can analyze larger volumes of data, and embed analytics in PowerPoint, making this solution easily integrated with Office.
• Some reviews report that extensive analysis functions are rarely usable without additional user training. In addition, the costs for the individual modules are relatively high and there are comparatively few feature updates.

Domo

Domo is a BI solution comprised of multiple systems that are featured in this platform, starting with connecting the data, and finishing with extending data with pre-built and custom apps from the Domo Appstore. You can use Domo also for your data lakes, warehouses, and ETL tools, alongside with R or Python scripts to prepare data for predictive modeling. Similar to other tools, you can connect the data across your enterprise, utilize their machine learning and artificial intelligence capabilities while enabling users to explore the data on their own.

Key features:

Numerous pre-built cloud connectors: Data integration works similarly to other tools, as mentioned; you can connect to more than 1000 pre-built cloud connectors, on-premise with Domo Workbench, file upload such as Excel or Google Docs, proprietary systems, and federated connectors that will enable you to query the data where it’s hosted.

Magic ETL feature: Data transformation tools will enable you to prepare your data with a drag-and-drop Magic ETL, as Domo calls it, without the need to learn SQL.

Automatically suggested visualizations: Data visualization enables you to go from raw data to charts, graphs, maps, and other visualizations which are suggested automatically by Domo after you upload your data source. You can refine the data, annotate for further commentary or discussions, or refine and control who has access to it.

Mr. Roboto as an AI engine: Predictive analysis is powered with Mr. Roboto, Domo’s AI engine, that enables data scientists to create predictive models and utilize artificial intelligence, machine learning, natural language processing, and other AI technologies that alert and notify about changes in KPIs, for example.

Domo Appstore: Their open cloud platform is an ecosystem of pre-built apps, connector APIs and dashboards, giving you also the possibility to build your own and share it on the Domo Appstore.

Domo is not built for newcomers as the learning curve can be steep, but they do offer trainings, therefore, be prepared to learn. Their pricing models are not transparent, but you can assume it’s the most expensive solution on our list, according to the available information on the web.

Yellowfin BI

Yellowfin BI – a suite of products consisted of dashboards, signals, stories, data discovery and data prep, this BI analytics tool offers numerous features, including a mobile app available both for Android and iOS devices.

Key features:

6. 5 main products: The Yellowfin BI suite combines 5 main products to connect, transform and understand your company data: Yellowfin signals, stories, dashboards, data discovery, and data prep.
7. 3 areas of BI and analytics: They are specialized in 3 main areas of BI and analytics solutions: enterprise analytics, embedded analytics, and external analytical apps.
8. Yellowfin signals via smartphone: Benefit from the automated detection of trends and anomalies in your data thanks to the Yellowfin signals, which are based on powerful, AI-controlled algorithms and enable immediate, personalized notifications directly to your smartphone.
9. Persuasive data stories: Yellowfin stories enable users to combine data with a written narrative, video, and images to build trusted data reports, that put people first.
10. Smart tasks: They offer trigger-based smart tasks that are automatically sent to the relevant person if a KPI falls below a standard. This enables everyone within the company to take the right action at the right time.

Some users wrote in their reviews that some essential analysis functions are not available that can be found in other BI tools. They also reported performance problems when dealing with larger data sets.

 IBM Cognos Analytics

IBM Cognos Analytics is a cloud-based business intelligence software that utilizes AI recommendations when creating dashboards and reports, geospatial capabilities to overlay your data with the physical world, and enables you to ask questions in plain English to communicate with the software.

Key features:

Search mechanism: A new Cognos Analytics search mechanism that enables users to access, discover and access recently used content within the software, where you can filter and save your searches.

A single data module: Multiple data sources are joined into a single data module that doesn’t replace the IT department but allows users to generate and work with insights on their own.

Interactive data visualization: Interactive visualizations are coupled with embedded geospatial mapping and advanced analytics.

AI assistant: Enables users to communicate in plain language: simply ask a question, and the assistant will recommend the most suitable visualizations.

Extensive knowledge center: Integration with other applications: A comprehensive knowledge center, community, and support for learning and understanding their product.

It will take a while to get familiar with all the features and capabilities this solution has on offer, and if you look for comprehensive dashboard features, this tool would not be suitable, as many users reported.

GoodData

GoodData is a business analytics software that provides the tools for data ingestion, storage, analytic queries, visualizations, and application integration. You can embed their analytics into your website, desktop or mobile application or create dashboards and reports for your daily activities, without the need to obtain a Ph.D., as stated on their website. Also one of the younger software solutions on our list, founded in 2007, GoodData received a stable 4 stars review on Capterra as one of the great tools to “manage your business.”

Key features:

• Customers can publish their own reports: Embedded analytics can be delivered at scale, for each customer you might have, with a wide range of customizations (custom data models and analytics) that let customers publish their own reports and dashboards.
• A modular data pipeline: They offer also modular data pipeline for fast data integration with an on-premise or cloud warehouse, no matter if the data is structured, semi-structured or unstructured. You can also use tools such as SQL, Python, R or Ruby for flexible data processing.
• A platform for developers: An open platform for developers provides you with frameworks to integrate with the existing data by utilizing white-labeled iFrames or their JavaScript library supported by numerous React components.
• Additional support: They offer additional services and support for the complete data and analytics lifecycle, including development, testing, launch, and maintenance.
• 4 Data centers: GoodData has 4 different data centers: two in the USA (Chicago and Dallas) and one in the EU and Canada, although the exact locations in these two regions are not specified. Existing projects can be transferred from one data center to another at any time.

Some users report it’s difficult to use despite completed training sessions and a higher price range for small businesses.

MicroStrategy

MicroStrategy is an enterprise analytics and mobility platform focused on hyperintelligence, federated analytics, and cloud solutions. Their mobile dossiers enable users to build interactive books of analytics that render on iOS or Android devices, with the possibility to extend the MicroStrategy content into their apps by using Xcode or JavaScript. You can download a sample app, MicroStrategy Mobile, or the Library app where you will enable you to deploy analytics on-the-go.

Key features:

• Hyperintelligence pulls your data: Their hyperintelligence feature pulls the data from various sources and puts them into relevant, zero-click insights, whether it’s a card on a website or integrated with a voice-over assistant. An interesting fact is that they provide contextual insights, you simply need to hover with your mouse, and the software will recognize the relevant data, directly in the website, applications, and devices you use.
• Federated analytics: It provides a common data glossary, no matter the tools you use, built on an enterprise semantic graph that indexes all the disparate data systems and enables visualizations of the same data.
• Mobile deployment: As mentioned, they also offer mobile deployment that enables users to leverage existing reports, dashboards and visualizations in real-time, from any location and device, with the possibility for code-free app development and design.
• Integration with voice technology: Powered by natural language processing and machine learning, you can also integrate voice technology and chatbots such as Alexa or Google Home.
• Cloud technology: Microstrategy offers a range of solutions to deliver scalable, secure, flexible and extensible cloud solutions that meet the needs of both business departments and IT.
• Some users report a complex initial setup since it involves many parties and in-depth knowledge of the application itself. Ease of use is certainly an aspect that the software lacks – you will need very knowledgeable staff to “get it up and running.”

ActivTrak

ActivTrak from Birch Grove Software is a flexible BI tool for team behavior analytics. The full system can handle teams from five to 1,000 and is designed for business owners, IT and HR managers, and team leaders who want to track their team’s productivity.

This option can be deployed via the cloud, SaaS, and the web, and offers mobile applications for both iOS and Android. Support for ActivTrak is available online and via phone during business hours.

Features:

• With ActivTrak, companies can monitor their employees without them being aware of it. The application is invisible in the monitored computer’s system, therefore staff can remain unaware that their computer usage is being watched. This way, employers gain an unadulterated insight on their staffs’ behaviors.
• ActivTrak provides employers and administrators with a complete log of activities done on a unit. The application does this by keeping track of active applications and takes note when there are changes. Relevant information is included, such as which programs were deployed, how much time was spent on each application, and more.
• To provide further information, ActivTrak takes continuous screenshots of computer screens. This action is automated so the business owner, the IT department, or the administrators need not manually trigger the screen capture feature. This lets companies connect activities logged in the audit trail and with the screenshots taken at the time it was noted by the system.
• ActivTrak agents can view users’ computer activities real-time. They can watch a live stream of active windows on all monitored machines. On top of that, the software also provides relevant information from the window being used by an employee on a device under surveillance.
• With ActivTrak, companies can disable the use of websites they deem to be disruptive during work hours. They are also able to utilize this feature to ensure that staff do not visit websites that may be harmful to the company’s IT resources. This way, the business is protected from possible malicious infiltrations.
• ActivTrak comes with seven types of reports that let employers gain a comprehensive overview of their company productivity. With this feature, they can see which websites are the most visited, the applications most often used, and how employees go about their work tasks.
• ActivTrak confers upon the employers and their agents the ability to access raw data anytime. They are able to do this because of the application’s capability to integrate with data warehouse infrastructures.Furthermore, they can utilize this feature to connect to BI tools to synchronize with other business intelligence software.

ChartMogul

ChartMogul boasts well-known clients such as Typeform and Docker, which speaks to its ability to handle larger user bases. This software is best for businesses who are trying to build or grow a subscription service, as it can measure churn rate and customer lifetime value.

ChartMogul can be deployed via the cloud, SaaS, and the web. Support is available 24/7 through a live representative and online tutorials. Representatives also offer live online training to help businesses get started.

For one, users can easily review and analyze meaningful metrics. This provides you with deeper insights that enable them to truly understand the performance of your business. ChartMogul makes specifying metrics needed to analyze your subscription revenue a tad easier. With a more particular set of metrics, you can easily have a more intelligent breakdown of your company’s performance and determine areas that require attention and improvement.

ChartMogul also helps in quickly identifying the factors that cause customers to upgrade, downgrade and churn and focus on addressing issues that impede business growth. Another benefit ChartMogul provides is the ability to sync subscription data in real-time so that metrics and billing systems are always updated.

Cluvio

Cluvio is for businesses that have a staff member with a bit of data analytics experience. This software allows businesses to run SQL queries against their databases, can process data in R, and turns out some impressive-looking dashboards and data visualizations.

Cluvio can be deployed via the cloud, SaaS, and the web, and support is offered online or phone during business hours. If businesses need help getting started, the vendor offers webinars and live online training, as well as training materials.

Key features include a SQL editor, customizable R scripts, push notifications, dashboard sharing with clients and more. The Cluvio dashboard can also be shared with the colleagues through a link which enables users to access it without logging in. Users to slice and dice data, specify date ranges and configure aggregation settings.

Cluvio offers integration with databases including PostgreSQL, MySQL, Amazon Redshift, Google BigQuery, MariaDB, Microsoft SQLServer, Amazon Web Services, Oracle, Google Cloud Platform, Amazon Aura, and Exasol.

Databox

Databox is the powerhouse of free BI software, scoring high with reviewers in ease-of-use, functionality, and customer support.

Databox allows businesses to track and report on their KPIs through a visually pleasing dashboard, can deliver information across multiple devices, and integrates with popular systems such as Adobe Analytics and Salesforce.Databox can be deployed via the cloud, SaaS, and the web, and offers mobile applications for iOS and Android. It is suitable for small to large businesses and offers mobile apps for iOS and Android devices.

Users can connect with various sources of cloud-based data such as HubSpot, Google Analytics, Instagram, Stripe, Facebook Ads and Moz. Users can also import data from their own databases using Databox’s REST API and SDKs. The Databox mobile app provides real-time access to company performance in the form of daily scorecards, weekly summaries and alerts.

Kyubit Business Intelligence

Kyubit Business Intelligence by Kyubit Solutions is a flexible BI tool that allows businesses to create dashboards and reports from their OLAP, SQL, and CSV data. Although this tool is easy to use, the setup makes it best-suited for those who already have some data analytics knowledge or business intelligence training.

Kyubit can be deployed via the cloud, SaaS, the web, or as an installed software solution. Support is available online or during business hours. For the non-tech-savvy user, Kyubit offers training documents to guide you through the software.

Kyubit is a cloud-based and on-premise business intelligence platform designed for Microsoft SQL Server Analysis Services (SSAS) environments. Primary features include online analytical processing (OLAP) grid and chart analysis and dashboard visualization.

Kyubit can be deployed via the cloud, SaaS, the web, or as an installed software solution. Support is available online or during business hours. For the non-tech-savvy user, Kyubit offers training documents to guide you through the…
Click To Tweet

Kyubit features a drag-and-drop interface for creating OLAP visualizations. The system offers grid, chart and report-style views of the results of OLAP analysis. Users can also export results in Excel and PDF formats and share them with other users in the corporate intranet.

Dashboards enable users to provide overviews of key performance indicators (KPIs) and scorecards using SQL and OLAP database. Dashboard elements include OLAP analysis capabilities such as slice, drill down and drill through. Additionally, dashboard elements can be based on multidimensional expressions (MDX) and T-SQL queries.

The post 20 Best Business Intelligence Software: Features And Application appeared first on Quantum: Machine Learning & Analytics.

Google AutoML is a tool from Google that automates the process of developing machine learning algorithms for various tasks. It’s user-friendly, fairly simple to use, and cloud-based.

AutoML is implemented in five different applications:

• Vision
• Video Intelligence
• Natural Language
• Translation
• Tables

Neural Architecture Search implemented as a RNN that generates architectures encoded as a variable-length sequence of tokens. The sequence of tokens generated can be seen as a sequence of actions that should be done to generate the architecture.

Once an architecture is generated, the proposed model is constructed and trained to finally record the acquired accuracy. The RNN is trained used reinforcement learning with a policy that updates the parameters of the RNN so that it generates better architectures over time.

Google Cloud AutoML enables developers with limited machine learning expertise to train high-quality models specific to their business needs.
Click To Tweet

Cloud AutoML enables developers with limited machine learning expertise to train high-quality models specific to their business needs. It relies on Google’s state-of-the-art transfer learning and neural architecture search technology.

AutoML Workflow

The businesses can bring their dataset, identify labels, and set the button. The steps that take place between data acquisition and prediction is abstracted by the AutoML platform. The trained and optimized machine learning model predicts. In brief, most of the steps are handled behind the scene providing time and opportunity for businesses to stay focused on problem and solutions rather than process and workflow.

Many of the AutoML platforms support to export the trained model into mobile devices running in iOS and Android, enabling developers to integrate easily within their mobile devices. As they get exported into the Docker container, DevOps team can deploy them at scale and infer in production environments.

AutoML Vision

AutoML Vision enables you to train machine learning models to classify your images according to own defined labels.

• Train models from labeled images and evaluate their performance.
• Leverage a human labeling service for datasets with unlabeled images.
• Register trained models for serving through the AutoML API.

Cloud Vision includes several options to use in order to integrate machine learning vision models into  applications and web sites.

AutoML Vision Edge now allows also to export custom trained models.

• AutoML Vision Edge allows to train and deploy low-latency, high accuracy models optimized for edge devices.
• With Tensorflow Lite, Core ML, and container export formats, AutoML Vision Edge supports a variety of devices.
• Hardware architectures supported: Edge TPUs, ARM and NVIDIA.

Cloud AutoML Vision Object Detection enables developers to train custom machine learning models that are capable of detecting individual objects in a given image along with its bounding box and label.

Cloud Vision Product Search allows retailers to create products, each containing reference images that visually describe the product from a set of viewpoints. Retailers can then add these products to product sets. Currently Vision API Product Search supports the following product categories: homegoods, apparel, toys, packaged goods, and general .

To build an application on iOS or Android devices use AutoML Vision Edge in ML Kit. This solution is available via Firebase and offers an end-to-end development flow for creating and deploying custom models to mobile devices using ML Kit client libraries.

AutoML Video Intelligence

Video Intelligence includes several options for integrating machine learning video intelligence models into your applications and web sites.

AutoML Video Intelligence Classification enables you to train machine learning models to classify shots and segments in your videos according to your own defined labels.

AutoML Video Intelligence Object Tracking enables you to train machine learning models to detect and track multiple objects in shots and segments. You can use these models to track objects in your videos according to your own pre-defined, custom labels.

AutoML Natural Language

AutoML Natural Language enables building and deploying custom machine learning models that analyze documents, categorizing them, identify entities within them, or assessing attitudes within them.

If you don’t need a custom model solution, the Cloud Natural Language API provides content classification, entity and sentiment analysis, and more.

AutoML Natural Language enables you to build and deploy custom machine learning models that analyze documents, categorizing them, identify entities within them, or assessing attitudes within them.
Click To Tweet

AutoML Natural Language uses machine learning to analyze the structure and meaning of documents. You train a custom machine learning model to classify documents, extract information, or understand the sentiment of authors.

•  A classification model analyzes a document and returns a list of content categories that apply to the text found in the document.
•  An entity extraction model inspects a document for known entities referenced in the document and labels those entities in the text.
•  A sentiment analysis model inspects a document and identifies the prevailing emotional opinion within it, especially to determine a writer’s attitude as positive, negative, or neutral.

Basic classification, entity extraction, and sentiment analysis are available through the Cloud Natural Language API. AutoML Natural Language enables defining custom classification categories, entities, and sentiment scores that are relevant to your application.

AutoML Translation

AutoML Translation allows to create own, custom translation models so that translation queries return results specific to the domain.

AutoML Translation trains custom models using matching pairs of sentences in the source and target languages. It treats each sentence pair as an independent training item, without assuming any correlation between separate pairs.

The sentence pairs used to train your custom model must be in Tab-separated values (.tsv) or Translation Memory eXchange (.tmx) format. It is possible to batch multiple .tsv and .tmx files into a comma-separated values (.csv) file.

Translation Memory eXchange (TMX) is a standard XML format for providing source and target translation sentences. AutoML Translation supports input files in a format based on TMX version 1.4. This example illustrates the required structure:

AutoML Tables

AutoML Tables enables entire team of data scientists, analysts, and developers to automatically build and deploy state-of-the-art machine learning models on structured data at massively increased speed and scale. Transform your business by leveraging your enterprise data to tackle mission-critical tasks like supply chain management, fraud detection, lead conversion optimization, and increasing customer lifetime value.

AutoML Tables is a supervised learning service. This means that you train a machine learning model with example data. AutoML Tables uses tabular (structured) data to train a machine learning model to make predictions on new data. One column from the dataset, called the target, is what your model will learn to predict. Some number of the other data columns are inputs (called features) that the model will learn patterns from. You can use the same input features to build multiple kinds of models just by changing the target. From the email marketing example, this means you could build two models with the same input features: One model could predict a customer’s persona (a categorical target), and one could predict their monthly spending (a numerical target).

AutoML Tables enables entire team of data scientists, analysts, and developers to automatically build and deploy state-of-the-art machine learning models on structured data at massively increased speed and scale.
Click To Tweet

Google Auto-ML advantages

Google’s AutoML efforts aim to make ML more scalable and accelerate both research and industry applications. Initial efforts of neural architecture search have enabled breakthroughs in computer vision with NasNet, and evolutionary methods such as AmoebaNet and hardware-aware mobile vision architecture MNasNet further show the benefit of these learning-to-learn methods.

Recently a learning-based approach was applied to tabular data, creating a scalable end-to-end AutoML solution that meets three key criteria:

• Full automation: Data and computation resources are the only inputs, while a servable TensorFlow model is the output. The whole process requires no human intervention.
• Extensive coverage: The solution is applicable to the majority of arbitrary tasks in the tabular data domain.
• High quality: Models generated by AutoML has comparable quality to models manually crafted by top ML experts.

About Google

Google LLC is an American multinational technology company that specializes in Internet-related services and products, which include online advertising technologies, a search engine, cloud computing, software, and hardware.The company’s rapid growth since incorporation has triggered a chain of products, acquisitions, and partnerships beyond Google’s core search engine (Google Search). It offers services designed for work and productivity (Google Docs, Google Sheets, and Google Slides), email (Gmail), scheduling and time management (Google Calendar), cloud storage (Google Drive), instant messaging and video chat (Duo, Hangouts, Meet), language translation (Google Translate), mapping and navigation (Google Maps, Waze, Google Earth, Street View), video sharing (YouTube), note-taking (Google Keep), and photo organizing and editing (Google Photos).

Read more about H2O Driverless AI, Data Science Machine, Azure AutoML and other automated machine learning software.

Subscribe to get latest updates:

https://twitter.com/sakalosch

The post Google AutoML appeared first on Quantum: Machine Learning & Analytics.

Azure Auto-ML

Azure Automated Machine Learning  –  cloud-based environment for  training, deploying, automating, managing and tracking ML models. Automated machine learning picks an algorithm and hyperparameters for you and generates a model ready for deployment.

Automated ML democratizes the machine learning model development process, and empowers its users, no matter their data science expertise, to identify an end-to-end machine learning pipeline for any problem.

Data scientists, analysts, and developers across industries can use automated ML to:

• Implement ML solutions without extensive programming knowledge
• Save time and resources
• Leverage data science best practices
• Provide agile problem-solving

Benefits for companies:

• Automatically build and deploy predictive models using the no-code UI or through a code-first notebooks experience.
• Increase productivity with easy data exploration and profiling and with intelligent feature engineering.
• Easily create accurate models customized to your data and refined by a wide array of algorithms and hyperparameters.
• Build responsible AI solutions with model interpretability, and fine-tune your models to improve accuracy.

Azure  AutoML: classify, regression, & forecast

Azure Automated Machine Learning could be applied to classifications, regressions and  forecasting. Classification is a common machine learning task. Azure Machine Learning offers featurizations specifically for these tasks, such as deep neural network text featurizers for classification. Common classification examples include fraud detection, handwriting recognition, and object detection.

Regression tasks are also a common supervised learning task. Azure Machine Learning offers featurizations specifically for these tasks. Different from classification where predicted output values are categorical, regression models predict numerical output values based on independent predictors. In regression, the objective is to help establish the relationship among those independent predictor variables by estimating how one variable impacts the others. For example, automobile price based on features like, gas mileage, safety rating, etc.

Time-series forecasting is an integral part of any business, whether it’s revenue, inventory, sales, or customer demand. Automated ML can be used to combine techniques and approaches and get a recommended, high-quality time-series forecast.

Automated ML can be used to time-series forecasting, which is an integral part of any business, whether it's revenue, inventory, sales, or customer demand.
Click To Tweet

An automated time-series experiment is treated as a multivariate regression problem. Past time-series values are “pivoted” to become additional dimensions for the regressor together with other predictors. This approach, unlike classical time series methods, has an advantage of naturally incorporating multiple contextual variables and their relationship to one another during training. Automated ML learns a single, but often internally branched model for all items in the dataset and prediction horizons. More data is thus available to estimate model parameters and generalization to unseen series becomes possible.

Advanced forecasting configuration includes:

• holiday detection and featurization
• time-series and DNN learners (Auto-ARIMA, Prophet, ForecastTCN)
• many models support through grouping
• rolling-origin cross validation
• configurable lags

AutoML workflow

During training, Azure Machine Learning creates a number of pipelines in parallel that try different algorithms and parameters for you. The service iterates through ML algorithms paired with feature selections, where each iteration produces a model with a training score. The higher the score, the better the model is considered to “fit” the data. It will stop once it hits the exit criteria defined in the experiment.

Using Azure Machine Learning, design and run automated ML training experiments with these steps:

1. Identify the ML problem to be solved: classification, forecasting, or regression
2. Choose whether to use the Python SDK or the studio web experience:
3. Specify the source and format of the labeled training data: Numpy arrays or Pandas dataframe
4. Configure the compute target for model training, such as your local computer, Azure Machine Learning Computes, remote VMs, or Azure Databricks.
5. Configure the automated machine learning parameters that determine how many iterations over different models, hyperparameter settings, advanced preprocessing/featurization, and what metrics to look at when determining the best model.
6. Submit the training run.
7. Review the results

Model Configuration

There are several options that you can use to configure automated machine learning experiments.

• Configuration options available in automated machine learning:
• Select your experiment type: Classification, Regression, or Time Series Forecasting
• Data source, formats, and fetch data
• Choose your compute target: local or remote
• Automated machine learning experiment settings
• Run an automated machine learning experiment
• Explore model metrics
• Register and deploy model

Automated machine learning supports data that resides on your local desktop or in the cloud such as Azure Blob Storage. The data can be read into a Pandas DataFrame or an Azure Machine Learning TabularDataset.

Feature engineering

Feature engineering is the process of using domain knowledge of the data to create features that help ML algorithms learn better. In Azure Machine Learning, scaling and normalization techniques are applied to facilitate feature engineering. Collectively, these techniques and feature engineering are referred to as featurization.

Automated machine learning featurization steps (feature normalization, handling missing data, converting text to numeric, etc.) become part of the underlying model. When using the model for predictions, the same featurization steps applied during training are applied to your input data automatically.

In every automated machine learning experiment, data is automatically scaled or normalized to help algorithms perform well. During model training, one of the following scaling or normalization techniques will be applied to each model. Additional feature engineering techniques such as, encoding and transforms are also available.

Ensemble models

Automated machine learning supports ensemble models, which are enabled by default. Ensemble learning improves machine learning results and predictive performance by combining multiple models as opposed to using single models. The ensemble iterations appear as the final iterations of the run. Automated machine learning uses both voting and stacking ensemble methods for combining models:

• Voting: predicts based on the weighted average of predicted class probabilities (for classification tasks) or predicted regression targets (for regression tasks).
• Stacking: stacking combines heterogenous models and trains a meta-model based on the output from the individual models. The current default meta-models are LogisticRegression for classification tasks and ElasticNet for regression/forecasting tasks.

Azure Auto-ML supports ensemble models, which improve machine learning results and predictive performance by combining multiple models as opposed to using single models.
Click To Tweet

Many models

The Many Models Solution Accelerator (preview) builds on Azure Machine Learning and enables you to use automated ML to train, operate, and manage hundreds or even thousands of machine learning models.

For example, building a model for each instance or individual in the following scenarios can lead to improved results:

• Predicting sales for each individual store
• Predictive maintenance for hundreds of oil wells
• Tailoring an experience for individual users.

AutoML on Databricks

AutoML on Databricks automates Machine Learning pipelines from feature engineering, model search, hyperparameter tuning, and inference while providing data scientists with the flexibility and control they need.

Databricks automates various steps of the data science workflow including augmented data preparation, visualization, feature engineering, hyperparameter tuning, model search, and finally automatic model tracking, reproducibility, and deployment, through a combination of native product offerings, partnerships, and custom solutions for a fully controlled and transparent AutoML experience.

Benefits

• Scalability: Automatically scale up and down your workloads and speed up training time with out-of-the-box optimizations for the most popular ML frameworks.
• Control: Choose algorithms better suited for the task in either single node or multi-node environment, and limit the number of runs to keep costs down.
• Ease of use: Automatically log results with MLflow tracking and parallelize hyperparameter search with Hyperopt on Databricks.
• Unification: Run all AutoML steps on the same platform, from ETL to model training and inference, securely, collaboratively, and at scale.

Databricks automates various steps of the data science workflow including augmented data preparation, visualization, feature engineering, hyperparameter tuning, model search, and finally automatic model tracking, reproducibility, and…
Click To Tweet

Features

• MLflow Experiments Tracking
• Track, compare, and visualize hundreds of thousands of experiments using open source or Managed MLflow.
• Automated Hyperparameter Tuning for Distributed Machine Learning
• Deep integration with PySpark MLlib’s Cross Validation to automatically track MLlib experiments in MLflow.
• Automated Hyperparameter Tuning for Single-node Machine Learning
• Optimized and distributed hyperparameter search with enhanced Hyperopt and automated tracking to MLflow.
• Automated Model Search for Single-node Machine Learning
• Optimized and distributed conditional hyperparameter search with enhanced Hyperopt and automated tracking to MLflow.
• Databricks Labs AutoML Toolkit
• Automated end-to-end model building pipeline is available via Databricks Labs custom solutions.

AutoML & ONNX

With Azure Machine Learning,  use automated ML to build a Python model and have it converted to the ONNX format. Once the models are in the ONNX format, they can be run on a variety of platforms and devices.

The ONNX runtime also supports C#, so use the model built automatically in C# apps without any need for recoding or any of the network latencies that REST endpoints introduce.

About Microsoft Azure

Microsoft Azure  is a cloud computing service created by Microsoft for building, testing, deploying, and managing applications and services through Microsoft-managed data centers. It provides software as a service (SaaS), platform as a service (PaaS) and infrastructure as a service (IaaS) and supports many different programming languages, tools, and frameworks, including both Microsoft-specific and third-party software and systems.

Read more about automated machine learning here.

Here you can find inforamtion about Data Science Machine for automated machine learning from MIT.

Subscribe to get latest updates:

Tweets by sakalosch

The post Azure Auto-ML appeared first on Quantum: Machine Learning & Analytics.