publications

2023

1. IV 2023

   Unified Pedestrian Path Prediction Framework: A Comparison Study

   Lemmens, Jarl, Bighashdel, Ariyan,  Jancura, Pavol and 1 more author

   In 2023 IEEE Intelligent Vehicles Symposium, IV 2023 Jul 2023
2. OptLearnMAS ’23

   Coordinating Fully-Cooperative Agents Using Hierarchical Learning Anticipation

   Bighashdel, Ariyan, de Geus, Daan,  Jancura, Pavol and 1 more author

   arXiv Mar 2023

   Abs

   Learning anticipation is a reasoning paradigm in multi-agent reinforcement learning, where agents, during learning, consider the anticipated learning of other agents. There has been substantial research into the role of learning anticipation in improving cooperation among self-interested agents in general-sum games. Two primary examples are Learning with Opponent-Learning Awareness (LOLA), which anticipates and shapes the opponent’s learning process to ensure cooperation among self-interested agents in various games such as iterated prisoner’s dilemma, and Look-Ahead (LA), which uses learning anticipation to guarantee convergence in games with cyclic behaviors. So far, the effectiveness of applying learning anticipation to fully-cooperative games has not been explored. In this study, we aim to research the influence of learning anticipation on coordination among common-interested agents. We first illustrate that both LOLA and LA, when applied to fully-cooperative games, degrade coordination among agents, causing worst-case outcomes. Subsequently, to overcome this miscoordination behavior, we propose Hierarchical Learning Anticipation (HLA), where agents anticipate the learning of other agents in a hierarchical fashion. Specifically, HLA assigns agents to several hierarchy levels to properly regulate their reasonings. Our theoretical and empirical findings confirm that HLA can significantly improve coordination among common-interested agents in fully-cooperative normal-form games. With HLA, to the best of our knowledge, we are the first to unlock the benefits of learning anticipation for fully-cooperative games.

3. Off-Policy Action Anticipation in Multi-Agent Reinforcement Learning

   Bighashdel, Ariyan, Geus, Daan,  Jancura, Pavol and 1 more author

   CoRR Mar 2023

   Abs

   Learning anticipation in Multi-Agent Reinforcement Learning (MARL) is a reasoning paradigm where agents anticipate the learning steps of other agents to improve cooperation among themselves. As MARL uses gradient-based optimization, learning anticipation requires using Higher-Order Gradients (HOG), with so-called HOG methods. Existing HOG methods are based on policy parameter anticipation, i.e., agents anticipate the changes in policy parameters of other agents. Currently, however, these existing HOG methods have only been applied to differentiable games or games with small state spaces. In this work, we demonstrate that in the case of non-differentiable games with large state spaces, existing HOG methods do not perform well and are inefficient due to their inherent limitations related to policy parameter anticipation and multiple sampling stages. To overcome these problems, we propose Off-Policy Action Anticipation (OffPA2), a novel framework that approaches learning anticipation through action anticipation, i.e., agents anticipate the changes in actions of other agents, via off-policy sampling. We theoretically analyze our proposed OffPA2 and employ it to develop multiple HOG methods that are applicable to non-differentiable games with large state spaces. We conduct a large set of experiments and illustrate that our proposed HOG methods outperform the existing ones regarding efficiency and performance.

4. ReScience C

   [Re] Object Detection Meets Knowledge Graphs

   Lemmens, Jarl,  Jancura, Pavol,  Dubbelman, Gijs and 1 more author

   Mar 2023
5. IEEE RAL

   Continual Pedestrian Trajectory Learning With Social Generative Replay

   Wu, Ya, Bighashdel, Ariyan,  Chen, Guang and 2 more authors

   IEEE Robotics and Automation Letters Feb 2023

   Abs

   Learning to predict the trajectories of pedestrians is essential for improving safety and efficiency of mobile robots. The prediction is challenging since the robot needs to operate in multiple environments in which the motion patterns of pedestrians are different between environments. Existing pedestrian trajectory prediction models heavily rely on the availability of representative data samples during training. In the presence of additional training data from a new environment, these models must be retrained on all datasets to avoid catastrophic forgetting of the knowledge obtained from the already supported environments. In this paper, we address this catastrophic forgetting problem in the context of learning to predict the trajectories of pedestrians. We propose a pseudo-rehearsal approach based on a novel Generative Replay (GR) model, referred to as Social-GR . The proposed method is consistent with crowd motion patterns and is free of any explicit reference to past experiences. To demonstrate the problem of catastrophic forgetting and evaluate our solution, we develop the Continual Trajectory Prediction Benchmark , which consists of four tasks, each representing a real-world pedestrian trajectory dataset from a different environment. By conducting several experiments, we show that our proposed Social-GR approach significantly outperforms other continual learning methods that depend on explicit experience replay, including the state-of-the-art conditional-GR model. We further illustrate the robustness of our proposed approach to mitigating catastrophic forgetting by switching the order of environments and employing a more complex prediction model.

2022

1. Machine Learning

   Model-free inverse reinforcement learning with multi-intention, unlabeled, and overlapping demonstrations

   Bighashdel, Ariyan,  Jancura, Pavol,  and Dubbelman, Gijs

   Machine Learning Nov 2022

   Abs

   In this paper, we define a novel inverse reinforcement learning (IRL) problem where the demonstrations are multi-intention, i.e., collected from multi-intention experts, unlabeled, i.e., without intention labels, and partially overlapping, i.e., shared between multiple intentions. In the presence of overlapping demonstrations, current IRL methods, developed to handle multi-intention and unlabeled demonstrations, cannot successfully learn the underlying reward functions. To solve this limitation, we propose a novel clustering-based approach to disentangle the observed demonstrations and experimentally validate its advantages. Traditional clustering-based approaches to multi-intention IRL, which are developed on the basis of model-based Reinforcement Learning (RL), formulate the problem using parametric density estimation. However, in high-dimensional environments and unknown system dynamics, i.e., model-free RL, the solution of parametric density estimation is only tractable up to the density normalization constant. To solve this, we formulate the problem as a mixture of logistic regressions to directly handle the unnormalized density. To research the challenges faced by overlapping demonstrations, we introduce the concepts of shared pair, which is a state-action pair that is shared in more than one intention, and separability, which resembles how well the multiple intentions can be separated in the joint state-action space. We provide theoretical analyses under the global optimality condition and the existence of shared pairs. Furthermore, we conduct extensive experiments on four simulated robotics tasks, extended to accept different intentions with specific levels of separability, and a synthetic driver task developed to directly control the separability. We evaluate the existing baselines on our defined problem and demonstrate, theoretically and experimentally, the advantages of our clustering-based solution, especially when the separability of the demonstrations decreases.

2. IV 2022

   Scene Spatio-Temporal Graph Convolutional Network for Pedestrian Intention Estimation

   Naik, Abhilash Y., Bighashdel, Ariyan,  Jancura, Pavol and 1 more author

   In 2022 IEEE Intelligent Vehicles Symposium, IV 2022 Jul 2022

   Abs

   For safe and comfortable navigation of autonomous vehicles, it is crucial to know the pedestrian’s intention of crossing the street. Generally, human drivers are aware of the traffic objects (e.g., crosswalks and traffic lights) in the environment while driving; likewise, these objects would play a crucial role for autonomous vehicles. In this research, we propose a novel pedestrian intention estimation method that not only takes into account the influence of traffic objects but also learns their contribution levels on the intention of the pedestrian. Our proposed method, referred to as Scene SpatioTemporal Graph Convolutional Network (Scene-STGCN), takes benefits from the strength of Graph Convolutional Networks and efficiently encodes the relationships between the pedestrian and the scene objects both spatially and temporally. We conduct several experiments on the Pedestrian Intention Estimation (PIE) dataset and illustrate the importance of scene objects and their contribution levels in the task of pedestrian intention estimation. Furthermore, we perform statistical analysis on the relevance of different traffic objects in the PIE dataset and carry out an ablation study on the effect of various information sources in the scene. Finally, we demonstrate the significance of the proposed Scene-STGCN through experimental comparisons with several baselines. The results indicate that our proposed Scene-STGCN outperforms the current state-of-the-art method by 0.03 in terms of ROC-AUC metric.

2021

1. ECML/PKDD

   Deep Adaptive Multi-intention Inverse Reinforcement Learning

   Bighashdel, Ariyan, Meletis, Panagiotis,  Jancura, Pavol and 1 more author

   In Machine Learning and Knowledge Discovery in Databases. Research Track Jul 2021

   Abs

   This paper presents a deep Inverse Reinforcement Learning (IRL) framework that can learn an a priori unknown number of nonlinear reward functions from unlabeled experts’ demonstrations. For this purpose, we employ the tools from Dirichlet processes and propose an adaptive approach to simultaneously account for both complex and unknown number of reward functions. Using the conditional maximum entropy principle, we model the experts’ multi-intention behaviors as a mixture of latent intention distributions and derive two algorithms to estimate the parameters of the deep reward network along with the number of experts’ intentions from unlabeled demonstrations. The proposed algorithms are evaluated on three benchmarks, two of which have been specifically extended in this study for multi-intention IRL, and compared with well-known baselines. We demonstrate through several experiments the advantages of our algorithms over the existing approaches and the benefits of online inferring, rather than fixing beforehand, the number of expert’s intentions.

2013

1. PLoS ONE

   Composite survival index to compare virulence changes in Azole-resistant aspergillus fumigatus clinical isolates

   Mavridou, Eleftheria, Meletiadis, Joseph,  Jancura, Pavol and 6 more authors

   PLoS ONE Aug 2013

   Abs

   Understanding resistance to antifungal agents in Aspergillus fumigatus is of increasing importance for the treatment of invasive infections in immunocompromised patients. Although a number of molecular resistance mechanisms are described in detail, the potential accompanying virulence changes and impact on clinical outcome have had little attention. We developed a new measure of survival, the composite survival index (CSI) to use as a measure of the virulence properties of A. fumigatus. Using a novel mathematical model we found a strong correlation between the in vitro growth characteristics and virulence in vivo expressed as CSI. Our model elucidates how three critical parameters (the lag phase (τ), decay constant (λ), and growth rate (ν)) interact with each other resulting in a CSI that correlated with virulence. Hence, strains with a long lag phase and high decay constant were less virulent in a murine model of invasive aspergillosis, whereas high virulence for isolates with a high CSI was associated in vitro with rapid growth and short lag phases. Resistant isolates with cyp51A mutations, which account for the majority of azole resistant aspergillosis cases, did not show a lower virulence compared to azole-susceptible isolates. In contrast, the CSI index revealed that a non-cyp51A-mediated resistance mechanism was associated with a dramatic decrease in CSI. Because of its predictive value, the mathematical model developed may serve to explore strain characteristics in vitro to predict virulence in vivo and significantly reduce the number of experimental animals required in such studies. The proposed measure of survival, the CSI can be used more in a general form in survival studies to explore optimal treatment options.

2012

1. DEEN: a simple and fast algorithm for network community detection

   Jancura, Pavol, Mavroeidis, Dimitrios,  and Marchiori, Elena

   In Computational Intelligence Methods for Bioinformatics and Biostatistics - 8th International Meeting, CIBB 2011, Revised Selected Papers Dec 2012

   Abs

   This paper introduces an algorithm for network community detection called DEEN (Delete Edges and Expand Nodes) consisting of two simple steps. First edges of the graph estimated to connect different clusters are detected and removed, next the resulting graph is used for generating communities by expanding seed nodes. DEEN uses as parameters the minimum and maximum allowed size of a cluster, and a resolution parameter whose value influences the number of removed edges. Application of DEEN to the budding yeast protein network for detecting functional protein complexes indicates its capability to identify clusters containing proteins with the same functional category, improving on MCL, a popular state-of-the-art method for functional protein complex detection. Moreover, application of DEEN to two popular benchmark networks results in the detection of accurate communities, substantiating the effectiveness of the proposed method in diverse domains.

2. BMC Bioninf.

   A methodology for detecting the orthology signal in a PPI network at a functional complex level.

   Jancura, Pavol, Mavridou, Eleftheria,  Carrillo-de Santa Pau, Enrique and 1 more author

   BMC Bioinformatics Dec 2012

   Abs

   Stable evolutionary signal has been observed in a yeast protein-protein interaction (PPI) network. These finding suggests more connected regions of a PPI network to be potential mediators of evolutionary information. Because more connected regions of PPI networks contain functional complexes, we are motivated to exploit the orthology relation for identifying complexes that can be clearly attributed to such evolutionary signal. We proposed a computational methodology for detecting the orthology signal present in a PPI network at a functional complex level. Specifically, we examined highly functionally coherent putative protein complexes as detected by a clustering technique in the complete yeast PPI network, in the yeast sub-network which spans only ortholog proteins as determined by a given second organism, and in yeast sub-networks induced by a set of proteins randomly selected. We proposed a filtering technique for extracting orthology-driven clusters with unique functionalities, that is, neither enriched by clusters identified using the complete yeast PPI network nor identified using random sampling. Moreover, we extracted functional categories that can be clearly attributed to the presence of evolutionary signal as described by these clusters. Application of the proposed methodology to the yeast PPI network indicated that evolutionary information at a functional complex level can be retrieved from the structure of the network. In particular, we detected protein complexes whose functionality could be uniquely attributed to the evolutionary signal. Moreover, we identified functions that are over-represented in these complexes due the evolutionary signal.

2011

1. Describing the orthology signal in a PPI network at a functional, complex level

   Jancura, Pavol, Mavridou, Eleftheria,  Pontes, Beatriz and 1 more author

   In Bioinformatics Research and Applications - 7th International Symposium, ISBRA 2011, Proceedings May 2011

   Abs

   In recent work, stable evolutionary signal induced by orthologous proteins has been observed in a Yeast protein-protein interaction (PPI) network. This finding suggests more connected subgraphs of a PPI network to be potential mediators of evolutionary information. Because protein complexes are also likely to be present in such subgraphs, it is interesting to characterize the bias of the orthology signal on the detection of putative protein complexes. To this aim, we propose a novel methodology for quantifying the functionality of the orthology signal in a PPI network at a protein complex level. The methodology performs a differential analysis between the functions of those complexes detected by clustering a PPI network using only proteins with orthologs in another given species, and the functions of complexes detected using the entire network or sub-networks generated by random sampling of proteins. We applied the proposed methodology to a Yeast PPI network using orthology information from a number of different organisms. The results indicated that the proposed method is capable to isolate functional categories that can be clearly attributed to the presence of an evolutionary (orthology) signal and quantify their distribution at a fine-grained protein level.

2010

1. PRL

   Dividing protein interaction networks for modular network comparative analysis

   Jancura, Pavol,  and Marchiori, Elena

   Pattern Recognition Letters Oct 2010

   Abs

   The increasing growth of data on protein-protein interaction (PPI) networks has boosted research on their comparative analysis. In particular, recent studies proposed models and algorithms for performing network alignment, that is, the comparison of networks across species for discovering conserved functional complexes. In this paper, we present an algorithm for dividing PPI networks, prior to their alignment, into small sub-graphs that are likely to cover conserved complexes. This allows one to perform network alignment in a modular fashion, by acting on pairs of resulting small sub-graphs from different species. The proposed dividing algorithm combines a graph-theoretical property (articulation) with a biological one (orthology). Extensive experiments on various PPI networks are conducted in order to assess how well the sub-graphs generated by this dividing algorithm cover protein functional complexes and whether the proposed pre-processing step can be used for enhancing the performance of network alignment algorithms. Source code of the dividing algorithm is available upon request for academic use.

2008

1. Dividing protein interaction networks by growing orthologous articulations

   Jancura, Pavol, Heringa, Jaap,  and Marchiori, Elena

   In 3rd IAPR International Conference on Pattern Recognition in Bioinformatics, PRIB 2008 Dec 2008

   Abs

   The increasing growth of data on protein-protein interaction (PPI) networks has boosted research on their comparative analysis. In particular, recent studies proposed models and algorithms for performing network alignment, the comparison of networks across species for discovering conserved modules. Common approaches for this task construct a merged representation of the considered networks, called alignment graph, and search the alignment graph for conserved networks of interest using greedy techniques. In this paper we propose a modular approach to this task. First, each network to be compared is divided into small subnets which are likely to contain conserved modules. To this aim, we develop an algorithm for dividing PPI networks that combines a graph theoretical property(articulation) with a biological one (orthology). Next, network alignment is performed on pairs of resulting subnets from different species. We tackle this task by means of a state-of-the-art alignment graph model for constructing alignment graphs, and an exact algorithm for searching in the alignment graph. Results of experiments show the ability of this approach to discover accurate conserved modules, and substantiate the importance of the notions of orthology and articulation for performing comparative network analysis in a modular fashion.

2. Divide, align and full-search for discovering conserved protein complexes

   Jancura, Pavol, Heringa, Jaap,  and Marchiori, Elena

   In Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics - 6th European Conference, EvoBIO 2008, Proceedings Jul 2008

   Abs

   Advances in modern technologies for measuring protein-protein interaction (PPI) has boosted research in PPI networks analysis and comparison. One of the challenging problems in comparative analysis of PPI networks is the comparison of networks across species for discovering conserved modules. Approaches for this task generally merge the considered networks into one new weighted graph, called alignment graph, which describes how interaction between each pair of proteins is preserved in different networks. The problem of finding conserved protein complexes across species is then transformed into the problem of searching the alignment graph for subnetworks whose weights satisfy a given constraint. Because the latter problem is computationally intractable, generally greedy techniques are used. In this paper we propose an alternative approach for this task. First, we use a technique we recently introduced for dividing PPI networks into small subnets which are likely to contain conserved modules. Next, we perform network alignment on pairs of resulting subnets from different species, and apply an exact search algorithm iteratively on each alignment graph, each time changing the constraint based on the weight of the solution found in the previous iteration. Results of experiments show that this method discovers multiple accurate conserved modules, and can be used for refining state-of-the-art algorithms for comparative network analysis.