Exploiting petri-net structure for activity classification and user instruction within an industrial setting

Simon Worgan, Ardhendu Behera, Anthony Cohn, David Hogg

Research output: Contribution to conferencePaper

4 Citations (Scopus)

Abstract

Live workflow monitoring and the resulting user interaction in industrial settings faces a number of challenges. A formal workflow may be unknown or implicit, data may be sparse and certain isolated actions may be undetectable given current visual feature extraction technology. This paper attempts to address these problems by inducing a structural workflow model from multiple expert demonstrations. When interacting with a naive user, this workflow is combined with spatial and temporal information, under a Bayesian framework, to give appropriate feedback and instruction. Structural information is captured by translating a Markov chain of actions into a simple place/transition petri-net. This novel petri-net structure maintains a continuous record of the current workbench configuration and allows multiple sub-sequences to be monitored without resorting to second order processes. This allows the user to switch between multiple sub-tasks, while still receiving informative feedback from the system. As this model captures the complete workflow, human inspection of safety critical processes and expert annotation of user instructions can be made. Activity classification and user instruction results show a significant on-line performance improvement when compared to the existing Hidden Markov Model or pLSA based state of the art. Further analysis reveals that the majority of our model's classification errors are caused by small de-synchronisation events rather than significant workflow deviations. We conclude with a discussion of the generalisability of the induced place/transition petri-net to other activity recognition tasks and summarise the developments of this model.
Original languageEnglish
Pages113-120
DOIs
Publication statusPublished - 14 Nov 2011
EventAssociation for Computing Machinery (ACM) International Conference on Multimodal Interaction (ICMI) - Alicante, Spain
Duration: 14 Nov 201118 Nov 2011

Conference

ConferenceAssociation for Computing Machinery (ACM) International Conference on Multimodal Interaction (ICMI)
CountrySpain
CityAlicante
Period14/11/1118/11/11

Fingerprint

Petri nets
Feedback
Hidden Markov models
Markov processes
Feature extraction
Synchronization
Demonstrations
Inspection
Switches
Monitoring

Cite this

Worgan, S., Behera, A., Cohn, A., & Hogg, D. (2011). Exploiting petri-net structure for activity classification and user instruction within an industrial setting. 113-120. Paper presented at Association for Computing Machinery (ACM) International Conference on Multimodal Interaction (ICMI), Alicante, Spain. https://doi.org/978-1-4503-0641-6/11/11
Worgan, Simon ; Behera, Ardhendu ; Cohn, Anthony ; Hogg, David. / Exploiting petri-net structure for activity classification and user instruction within an industrial setting. Paper presented at Association for Computing Machinery (ACM) International Conference on Multimodal Interaction (ICMI), Alicante, Spain.
@conference{2f8f98301d864c1e877899116c3ceba2,
title = "Exploiting petri-net structure for activity classification and user instruction within an industrial setting",
abstract = "Live workflow monitoring and the resulting user interaction in industrial settings faces a number of challenges. A formal workflow may be unknown or implicit, data may be sparse and certain isolated actions may be undetectable given current visual feature extraction technology. This paper attempts to address these problems by inducing a structural workflow model from multiple expert demonstrations. When interacting with a naive user, this workflow is combined with spatial and temporal information, under a Bayesian framework, to give appropriate feedback and instruction. Structural information is captured by translating a Markov chain of actions into a simple place/transition petri-net. This novel petri-net structure maintains a continuous record of the current workbench configuration and allows multiple sub-sequences to be monitored without resorting to second order processes. This allows the user to switch between multiple sub-tasks, while still receiving informative feedback from the system. As this model captures the complete workflow, human inspection of safety critical processes and expert annotation of user instructions can be made. Activity classification and user instruction results show a significant on-line performance improvement when compared to the existing Hidden Markov Model or pLSA based state of the art. Further analysis reveals that the majority of our model's classification errors are caused by small de-synchronisation events rather than significant workflow deviations. We conclude with a discussion of the generalisability of the induced place/transition petri-net to other activity recognition tasks and summarise the developments of this model.",
author = "Simon Worgan and Ardhendu Behera and Anthony Cohn and David Hogg",
year = "2011",
month = "11",
day = "14",
doi = "978-1-4503-0641-6/11/11",
language = "English",
pages = "113--120",
note = "Association for Computing Machinery (ACM) International Conference on Multimodal Interaction (ICMI) ; Conference date: 14-11-2011 Through 18-11-2011",

}

Worgan, S, Behera, A, Cohn, A & Hogg, D 2011, 'Exploiting petri-net structure for activity classification and user instruction within an industrial setting' Paper presented at Association for Computing Machinery (ACM) International Conference on Multimodal Interaction (ICMI), Alicante, Spain, 14/11/11 - 18/11/11, pp. 113-120. https://doi.org/978-1-4503-0641-6/11/11

Exploiting petri-net structure for activity classification and user instruction within an industrial setting. / Worgan, Simon; Behera, Ardhendu; Cohn, Anthony; Hogg, David.

2011. 113-120 Paper presented at Association for Computing Machinery (ACM) International Conference on Multimodal Interaction (ICMI), Alicante, Spain.

Research output: Contribution to conferencePaper

TY - CONF

T1 - Exploiting petri-net structure for activity classification and user instruction within an industrial setting

AU - Worgan, Simon

AU - Behera, Ardhendu

AU - Cohn, Anthony

AU - Hogg, David

PY - 2011/11/14

Y1 - 2011/11/14

N2 - Live workflow monitoring and the resulting user interaction in industrial settings faces a number of challenges. A formal workflow may be unknown or implicit, data may be sparse and certain isolated actions may be undetectable given current visual feature extraction technology. This paper attempts to address these problems by inducing a structural workflow model from multiple expert demonstrations. When interacting with a naive user, this workflow is combined with spatial and temporal information, under a Bayesian framework, to give appropriate feedback and instruction. Structural information is captured by translating a Markov chain of actions into a simple place/transition petri-net. This novel petri-net structure maintains a continuous record of the current workbench configuration and allows multiple sub-sequences to be monitored without resorting to second order processes. This allows the user to switch between multiple sub-tasks, while still receiving informative feedback from the system. As this model captures the complete workflow, human inspection of safety critical processes and expert annotation of user instructions can be made. Activity classification and user instruction results show a significant on-line performance improvement when compared to the existing Hidden Markov Model or pLSA based state of the art. Further analysis reveals that the majority of our model's classification errors are caused by small de-synchronisation events rather than significant workflow deviations. We conclude with a discussion of the generalisability of the induced place/transition petri-net to other activity recognition tasks and summarise the developments of this model.

AB - Live workflow monitoring and the resulting user interaction in industrial settings faces a number of challenges. A formal workflow may be unknown or implicit, data may be sparse and certain isolated actions may be undetectable given current visual feature extraction technology. This paper attempts to address these problems by inducing a structural workflow model from multiple expert demonstrations. When interacting with a naive user, this workflow is combined with spatial and temporal information, under a Bayesian framework, to give appropriate feedback and instruction. Structural information is captured by translating a Markov chain of actions into a simple place/transition petri-net. This novel petri-net structure maintains a continuous record of the current workbench configuration and allows multiple sub-sequences to be monitored without resorting to second order processes. This allows the user to switch between multiple sub-tasks, while still receiving informative feedback from the system. As this model captures the complete workflow, human inspection of safety critical processes and expert annotation of user instructions can be made. Activity classification and user instruction results show a significant on-line performance improvement when compared to the existing Hidden Markov Model or pLSA based state of the art. Further analysis reveals that the majority of our model's classification errors are caused by small de-synchronisation events rather than significant workflow deviations. We conclude with a discussion of the generalisability of the induced place/transition petri-net to other activity recognition tasks and summarise the developments of this model.

UR - http://dl.acm.org/citation.cfm?doid=2070481.2070502

UR - http://www.acm.org/icmi/2011/

U2 - 978-1-4503-0641-6/11/11

DO - 978-1-4503-0641-6/11/11

M3 - Paper

SP - 113

EP - 120

ER -

Worgan S, Behera A, Cohn A, Hogg D. Exploiting petri-net structure for activity classification and user instruction within an industrial setting. 2011. Paper presented at Association for Computing Machinery (ACM) International Conference on Multimodal Interaction (ICMI), Alicante, Spain. https://doi.org/978-1-4503-0641-6/11/11