Facilitating the spread of knowledge and innovation in professional software development

Parul Bavishi deconstructs the structure behind bestselling non-fiction titles, and looks at how to apply this to writing.

In this podcast, Matthew Skelton and Manuel Pais, co-authors of the book Team Topologies, sat down with InfoQ podcast co-host Daniel Bryant. Topics discussed included: the role of a modern software architect, how team design impacts software architecture, creating “team APIs” in order to reduce cognitive load, and the benefits of building a “thinnest viable platform”.

Oliver Strickson discusses automatic differentiation, a family of algorithms for taking derivatives of functions implemented by computer programs, offering the ability to compute gradients of values.

Tanya Spencer, Steph South discuss applying Agile techniques to exec and other non-technical teams to help them streamline and make real progress against their aspirations.

Sonya Moisset describes how GitHub Marketplace helped Pride in London automating and improving their workflow with different tools for accessibility, code coverage, code review, code quality, security and other functionalities (alerting with Slack). She talks about what OWASP is and how to improve the workflow for open source projects using GitHub Marketplace applications.

InfoQ Homepage


Microsoft Obtains Exclusive License for GPT-3 AI Model

Anthony Alford

Microsoft announced an agreement with OpenAI to license OpenAI’s GPT-3 deep-learning model for natural-language processing (NLP). Although Microsoft’s announcement says it has “exclusively” licensed the model, OpenAI will continue to offer access to the model via its own API.

Microsoft CTO Kevin Scott wrote about the agreement on Microsoft’s blog. The deal builds on an existing relationship between the two organizations, which includes a partnership in building a supercomputer on Microsoft’s Azure cloud platform. OpenAI recently used that supercomputer to train GPT-3, which at 175 billion parameters is one of the largest NLP deep-learning models trained to date. Scott said the licensing of GPT-3 will:

[Allow] us to leverage its technical innovations to develop and deliver advanced AI solutions for our customers, as well as create new solutions that harness the amazing power of advanced natural language generation.

GPT-3 is the third iteration of OpenAI’s Generative Pre-Trained Transformer model. The original GPT model was released in 2018 and contained 117 million parameters. For the next iteration, GPT-2, OpenAI scaled up the model more than 10x, to 1.5 billion parameters. Because the text generated by GPT-2 could often be as “credible” as text written by humans, OpenAI at first declined to release the full model, citing potential for misuse in generating “deceptive, biased, or abusive language at scale.” However, by November 2019, OpenAI had seen “no strong evidence of misuse” and decided to release the model.

In July 2019, Microsoft and OpenAI announced a partnership, which included a $1 billion investment from Microsoft, to “jointly build new Azure AI supercomputing technologies.” OpenAI also agreed to run its services on Azure and to make Microsoft its “preferred partner for commercializing new AI technologies.” During its Build conference this May, Microsoft showcased the supercomputer built for OpenAI on its Azure cloud platform: “a single system with more than 285,000 CPU cores, 10,000 GPUs and 400 gigabits per second of network connectivity for each GPU server.”

GPT-3, announced earlier this year, was a 100x scale-up of GPT-2 and set new state-of-the-art results on several NLP tasks. The training dataset contained nearly half a trillion words. Training the model on the Azure supercomputer consumed “several thousand petaflop/s-days of compute” and is estimated to have cost from $4.6 million to $12 million. As with GPT-2, OpenAI has not released the trained model; however, OpenAI did release a limited-access web API for developers to make calls to the model from their apps.

The licensing deal with Microsoft is the latest of several recent moves by OpenAI to monetize their technology. Originally founded as a non-profit, OpenAI launched a new “hybrid of a for-profit and nonprofit” or “capped-profit” called OpenAI LP in March 2019. The goal of the new company was to “raise investment capital and attract employees with startup-like equity.” OpenAI’s API page contains an FAQ section that defends its commercial products as “one of the ways to make sure we have enough funding to succeed.” While the terms of the Microsoft license have not been disclosed, OpenAI claims that it has “no impact” on users of OpenAI’s API, who can “continue building applications…as usual.”

With the license agreement being touted as “exclusive,” and given OpenAI’s past reluctance to release their trained models, many commenters have joked that the company should change its name to “ClosedAI.” One Hacker News reader questioned the long-term commercial viability of GPT-3:

Anyone else feel like this idea of commercializing GPT-3 is bound to go nowhere as the research community figures out how to replicate the same capabilities in smaller cheaper open models within a few months or even a year?

The OpenAI API is currently in beta, with a waitlist for gaining access. The 1.5-billion parameter GPT-2 model is available on GitHub.

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers.

View an example

You need to Register an InfoQ account or Login or login to post comments. But there’s so much more behind being registered.

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers.

View an example

A virtual conference for senior software engineers and architects on the trends, best practices and solutions leveraged by the world’s most innovative software shops.

InfoQ.com and all content copyright © 2006-2020 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we’ve ever worked with.

Privacy Notice, Terms And Conditions, Cookie Policy

Source: https://www.infoq.com/news/2020/09/microsoft-license-gpt-3/

Microsoft Corporation, OpenAI, Artificial intelligence

World news – CA – Microsoft Obtains Exclusive License for GPT-3 AI Model

En s’appuyant sur ses expertises dans les domaines du digital, des technologies et des process , CSS Engineering vous accompagne dans vos chantiers de transformation les plus ambitieux et vous aide à faire émerger de nouvelles idées, de nouvelles offres, de nouveaux modes de collaboration, de nouvelles manières de produire et de vendre.

CSS Engineering s’implique dans les projets de chaque client comme si c’était les siens. Nous croyons qu’une société de conseil devrait être plus que d’un conseiller. Nous nous mettons à la place de nos clients, pour aligner nos incitations à leurs objectifs, et collaborer pour débloquer le plein potentiel de leur entreprise. Cela établit des relations profondes et agréables.

Nos services:

  1. Création des sites web professionnels
  2. Hébergement web haute performance et illimité
  3. Vente et installation des caméras de vidéo surveillance
  4. Vente et installation des système de sécurité et d’alarme
  5. E-Marketing

Toutes nos réalisations ici https://www.css-engineering.com/en/works/

Previous articlePokemon Sword & Shield DLC Crown Tundra Coming In October
Next articleJet suit paramedic ‘could save lives’
Georges MOMO
Georges MOMO est Ingénieur informatique ( Diplôme obtenu à 3IL Limoges France) Il début sa carrière par un stage de fin d'étude d'ingénieur à Toulouse sur un projet de AIRBUS Toulouse Blagnac, il enchaîne ensuite à Nantes où il travaille sur le projet de la migration documentaire de la BPCE. Son ambition le conduit ensuite à Paris où il travaille sur deux projets de la banque de France (Validation des titres de bourse émis sur le marché européen et la génération centralisée des documents) qu'il quitte quelques années plus tard occupant le poste de responsable technique. Passionné de l'information, il est actif sur le web et sur le terrain depuis les années 2000.


Please enter your comment!
Please enter your name here