Daniele Castellana

Selected publications

See all

1. Conference

   The Infinite Contextual Graph Markov Model

   Daniele Castellana, Federico Errica, Davide Bacciu, and 1 more author

   In Proceedings of the 39th International Conference on Machine Learning, 17–23 jul 2022

   Abs Bib HTML PDF

   The Contextual Graph Markov Model (CGMM) is a deep, unsupervised, and probabilistic model for graphs that is trained incrementally on a layer-by-layer basis. As with most Deep Graph Networks, an inherent limitation is the need to perform an extensive model selection to choose the proper size of each layer’s latent representation. In this paper, we address this problem by introducing the Infinite Contextual Graph Markov Model (iCGMM), the first deep Bayesian nonparametric model for graph learning. During training, iCGMM can adapt the complexity of each layer to better fit the underlying data distribution. On 8 graph classification tasks, we show that iCGMM: i) successfully recovers or improves CGMM’s performances while reducing the hyper-parameters’ search space; ii) performs comparably to most end-to-end supervised methods. The results include studies on the importance of depth, hyper-parameters, and compression of the graph embeddings. We also introduce a novel approximated inference procedure that better deals with larger graph topologies.

   @inproceedings{castellana22,
     title = {The Infinite Contextual Graph {M}arkov Model},
     author = {Castellana, Daniele and Errica, Federico and Bacciu, Davide and Micheli, Alessio},
     booktitle = {Proceedings of the 39th International Conference on Machine Learning},
     pages = {2721--2737},
     year = {2022},
     volume = {162},
     series = {Proceedings of Machine Learning Research},
     month = {17--23 Jul},
     publisher = {PMLR},
   }

2. Journal

   A tensor framework for learning in structured domains

   Daniele Castellana, and Davide Bacciu

   Neurocomputing, 17–23 jul 2021

   Abs Bib HTML

   Learning machines for structured data (e.g., trees) are intrinsically based on their capacity to learn representations by aggregating information from the multi-way relationships emerging from the structure topology. While complex aggregation functions are desirable in this context to increase the expressiveness of the learned representations, the modelling of higher-order interactions among structure constituents is unfeasible, in practice, due to the exponential number of parameters required. Therefore, the common approach is to define models which rely only on first-order interactions among structure constituents. In this work, we leverage tensors theory to define a framework for learning in structured domains. Such a framework is built on the observation that more expressive models require a tensor parameterisation. This observation is the stepping stone for the application of tensor decompositions in the context of recursive models. From this point of view, the advantage of using tensor decompositions is twofold since it allows limiting the number of model parameters while injecting inductive biases that do not ignore higher-order interactions. We apply the proposed framework on probabilistic and neural models for structured data, defining different models which leverage tensor decompositions. The experimental validation clearly shows the advantage of these models compared to first-order and full-tensorial models.

   @article{Castellana2021neurocomp,
     title = {A tensor framework for learning in structured domains},
     journal = {Neurocomputing},
     year = {2021},
     issn = {0925-2312},
     doi = {https://doi.org/10.1016/j.neucom.2021.05.110},
     url = {https://www.sciencedirect.com/science/article/pii/S0925231221011164},
     author = {Castellana, Daniele and Bacciu, Davide},
     keywords = {Tensor decompositions, Structured data, Recursive neural models, Probabilistic models},
   }

3. Conference

   Learning from Non-Binary Constituency Trees via Tensor Decomposition

   Daniele Castellana, and Davide Bacciu

   In Proceedings of the 28th International Conference on Computational Linguistics, Dec 2020

   Abs Bib HTML PDF

   Processing sentence constituency trees in binarised form is a common and popular approach in literature. However, constituency trees are non-binary by nature. The binarisation procedure changes deeply the structure, furthering constituents that instead are close. In this work, we introduce a new approach to deal with non-binary constituency trees which leverages tensor-based models. In particular, we show how a powerful composition function based on the canonical tensor decomposition can exploit such a rich structure. A key point of our approach is the weight sharing constraint imposed on the factor matrices, which allows limiting the number of model parameters. Finally, we introduce a Tree-LSTM model which takes advantage of this composition function and we experimentally assess its performance on different NLP tasks.

   @inproceedings{Castellana2020coling,
     title = {Learning from Non-Binary Constituency Trees via Tensor Decomposition},
     author = {Castellana, Daniele and Bacciu, Davide},
     booktitle = {Proceedings of the 28th International Conference on Computational Linguistics},
     month = dec,
     year = {2020},
     publisher = {International Committee on Computational Linguistics},
     url = {https://aclanthology.org/2020.coling-main.346},
     doi = {10.18653/v1/2020.coling-main.346},
     pages = {3899--3910},
   }

4. Conference

   Bayesian Tensor Factorisation for Bottom-up Hidden Tree Markov Models

   Daniele Castellana, and Davide Bacciu

   In 2019 International Joint Conference on Neural Networks (IJCNN), Jul 2019

   Abs Bib HTML

   Bottom-Up Hidden Tree Markov Model is a highly expressive model for tree-structured data. Unfortunately, it cannot be used in practice due to the intractable size of its state-transition matrix. We propose a new approximation which lies on the Tucker factorisation of tensors. The probabilistic interpretation of such approximation allows us to define a new probabilistic model for tree-structured data. Hence, we define the new approximated model and we derive its learning algorithm. Then, we empirically assess the effective power of the new model evaluating it on two different tasks. In both cases, our model outperforms the other approximated model known in the literature.

   @inproceedings{Castellana2019b,
     author = {Castellana, Daniele and Bacciu, Davide},
     booktitle = {2019 International Joint Conference on Neural Networks (IJCNN)},
     doi = {10.1109/IJCNN.2019.8851851},
     isbn = {978-1-7281-1985-4},
     month = jul,
     pages = {1--8},
     publisher = {IEEE},
     title = {{Bayesian Tensor Factorisation for Bottom-up Hidden Tree Markov Models}},
     url = {https://ieeexplore.ieee.org/document/8851851/},
     volume = {2019-July},
     year = {2019},
   }

5. Journal

   Bayesian mixtures of Hidden Tree Markov Models for structured data clustering

   Davide Bacciu, and Daniele Castellana

   Neurocomputing, Jul 2019

   Advances in artificial neural networks, machine learning and computational intelligence

   Abs Bib HTML

   The paper deals with the problem of unsupervised learning with structured data, proposing a mixture model approach to cluster tree samples. First, we discuss how to use the Switching-Parent Hidden Tree Markov Model, a compositional model for learning tree distributions, to define a finite mixture model where the number of components is fixed by a hyperparameter. Then, we show how to relax such an assumption by introducing a Bayesian non-parametric mixture model where the number of necessary hidden tree components is learned from data. Experimental validation on synthetic and real datasets show the benefit of mixture models over simple hidden tree models in clustering applications. Further, we provide a characterization of the behaviour of the two mixture models for different choices of their hyperparameters.

   @article{Bacciu2019a,
     title = {Bayesian mixtures of Hidden Tree Markov Models for structured data clustering},
     journal = {Neurocomputing},
     volume = {342},
     pages = {49-59},
     year = {2019},
     note = {Advances in artificial neural networks, machine learning and computational intelligence},
     issn = {0925-2312},
     doi = {https://doi.org/10.1016/j.neucom.2018.11.091},
     url = {https://www.sciencedirect.com/science/article/pii/S0925231219301444},
     author = {Bacciu, Davide and Castellana, Daniele},
     keywords = {Hidden Tree Markov Models, Infinite mixtures, Dirichlet Process, Tree structured data}
   }