Uso de testes metamórficos para verificação de aplicação chatbot
Chatbots are software that make use of Artificial Intelligence (IA) to provide interaction between humans and machines through Natural Language (NL), resembling a conversation between two people. It is a current topic that has received a lot of attention in recent years. However, this type of app...
Na minha lista:
| Autor principal: | |
|---|---|
| Outros Autores: | |
| Formato: | Dissertação |
| Idioma: | pt_BR |
| Publicado em: |
Universidade Federal do Rio Grande do Norte
|
| Assuntos: | |
| Endereço do item: | https://repositorio.ufrn.br/handle/123456789/49527 |
| Tags: |
Adicionar Tag
Sem tags, seja o primeiro a adicionar uma tag!
|
| Resumo: | Chatbots are software that make use of Artificial Intelligence (IA) to provide interaction
between humans and machines through Natural Language (NL), resembling a conversation
between two people. It is a current topic that has received a lot of attention in recent years.
However, this type of application presents a behavior that is sometimes unpredictable, as
different users can request the same set of actions in different ways and the application can
return something different to each of the users. This makes the software testing activity
somewhat complex, since it is not simple to define the test oracle. The test oracle is the
process that verifies the test result, judging the response of the application, program or
function as correct or incorrect through specification and planning. In this scenario, an
appropriate testing technique to deal with the oracle problem is Metamorphic Testing (MT).
This technique was proposed as a solution to minimize the oracle definition problem, since
the application response verification process is performed by the definition of Metamorphic
Relations (MRs), elements that assume the role of the oracle. Metamorphic relationships
are properties of the test object application that define the behavior that the application
must maintain when submitted to a given test case. However, despite Metamorphic Testing
being a well-established technique to deal with the problem of the lack of a test oracle, it
is still little explored in the context of chatbots. Thus, the objective of this work was to
investigate the applicability and effectiveness of the MT technique in a chatbot application
developed by a public company. Therefore, it was necessary to develop a tool to automate
the test process by applying the MT concepts. For the evaluation of the technique we
selected 22 Source Test Cases (STCs) divided into 3 Intents and input this information
into the automation tool to run the tests of the 20 Metamorphic Relations. Through the
execution of the tool, it was possible to generate more than 4100 Follow-up Test Cases
(FTCs), derived from the STCs. The verification took place in two test batteries using
two versions of the same application with 3 months of development difference. With the
application of the MT technique, it was observed that 25% of the FTCs revealed defects,
representing more than 900 disapproved FTCs, per version tested. |
|---|