Towards Differential Privacy for Symbolic Systems
In this paper, we develop a privacy implementation for symbolic control systems. Such systems generate sequences of non-numerical data, and these sequences can be represented by words or strings over a finite alphabet. This work uses the framework of differential privacy, which is a statistical notion of privacy that makes it unlikely that privatized data will reveal anything meaningful about underlying sensitive data. To bring differential privacy to symbolic control systems, we develop an exponential mechanism that approximates a sensitive word using a randomly chosen word that is likely to be near it. The notion of ‘near’ is given by the Levenshtein distance, which counts the number of operations required to change one string into another. We then develop a Levenshtein automaton implementation of our exponential mechanism that efficiently generates privatized output words. This automaton has letters as its states, and this work develops transition probabilities among these states that give overall output words obeying the distribution required by the exponential mechanism. Numerical results are provided to demonstrate this technique for both strings of English words and runs of a deterministic transition system, demonstrating in both cases that privacy can be provided in this setting while maintaining a reasonable degree of accuracy.
Detecting Hate Speech and Offensive Language on Twitter using Machine Learning: An N-gram and TFIDF based Approach
Toxic online content has become a major issue in today’s world due to an exponential increase in the use of internet by people of different cultures and educational background. Differentiating hate speech and offensive language is a key challenge in automatic detection of toxic text content. In this paper, we propose an approach to automatically classify tweets on Twitter into three classes: hateful, offensive and clean. Using Twitter dataset, we perform experiments considering n-grams as features and passing their term frequency-inverse document frequency (TFIDF) values to multiple machine learning models. We perform comparative analysis of the models considering several values of n in n-grams and TFIDF normalization methods. After tuning the model giving the best results, we achieve 95.6% accuracy upon evaluating it on test data. We also create a module which serves as an intermediate between user and Twitter.
Data-Driven Design: Exploring new Structural Forms using Machine Learning and Graphic Statics
The aim of this research is to introduce a novel structural design process that allows architects and engineers to extend their typical design space horizon and thereby promoting the idea of creativity in structural design. The theoretical base of this work builds on the combination of structural form-finding and state-of-the-art machine learning algorithms. In the first step of the process, Combinatorial Equilibrium Modelling (CEM) is used to generate a large variety of spatial networks in equilibrium for given input parameters. In the second step, these networks are clustered and represented in a form-map through the implementation of a Self Organizing Map (SOM) algorithm. In the third step, the solution space is interpreted with the help of a Uniform Manifold Approximation and Projection algorithm (UMAP). This allows gaining important insights in the structure of the solution space. A specific case study is used to illustrate how the infinite equilibrium states of a given topology can be defined and represented by clusters. Furthermore, three classes, related to the non-linear interaction between the input parameters and the form space, are verified and a statement about the entire manifold of the solution space of the case study is made. To conclude, this work presents an innovative approach on how the manifold of a solution space can be grasped with a minimum amount of data and how to operate within the manifold in order to increase the diversity of solutions.
Learning to Read by Spelling: Towards Unsupervised Text Recognition
This work presents a method for visual text recognition without using any paired supervisory data. We formulate the text recognition task as one of aligning the conditional distribution of strings predicted from given text images, with lexically valid strings sampled from target corpora. This enables fully automated, and unsupervised learning from just line-level text-images, and unpaired text-string samples, obviating the need for large aligned datasets. We present detailed analysis for various aspects of the proposed method, namely – (1) the impact of the length of training sequences on convergence, (2) relation between character frequencies and the order in which they are learnt, and (3) demonstrate the generalisation ability of our recognition network to inputs of arbitrary lengths. Finally, we demonstrate excellent text recognition accuracy on both synthetically generated text images, and scanned images of real printed books, using no labelled training examples.
Envy-Free Classification
In classic fair division problems such as cake cutting and rent division, envy-freeness requires that each individual (weakly) prefer his allocation to anyone else’s. On a conceptual level, we argue that envy-freeness also provides a compelling notion of fairness for classification tasks. Our technical focus is the generalizability of envy-free classification, i.e., understanding whether a classifier that is envy free on a sample would be almost envy free with respect to the underlying distribution with high probability. Our main result establishes that a small sample is sufficient to achieve such guarantees, when the classifier in question is a mixture of deterministic classifiers that belong to a family of low Natarajan dimension.
Deformable Stacked Structure for Named Entity Recognition
Neural architecture for named entity recognition has achieved great success in the field of natural language processing. Currently, the dominating architecture consists of a bi-directional recurrent neural network (RNN) as the encoder and a conditional random field (CRF) as the decoder. In this paper, we propose a deformable stacked structure for named entity recognition, in which the connections between two adjacent layers are dynamically established. We evaluate the deformable stacked structure by adapting it to different layers. Our model achieves the state-of-the-art performances on the OntoNotes dataset.
Distributed Q-Learning for Dynamically Decoupled Systems
Matrix Linear Discriminant Analysis
We propose a novel linear discriminant analysis approach for the classification of high-dimensional matrix-valued data that commonly arises from imaging studies. Motivated by the equivalence of the conventional linear discriminant analysis and the ordinary least squares, we consider an efficient nuclear norm penalized regression that encourages a low-rank structure. Theoretical properties including a non-asymptotic risk bound and a rank consistency result are established. Simulation studies and an application to electroencephalography data show the superior performance of the proposed method over the existing approaches.
A Survey of Conventional and Artificial Intelligence / Learning based Resource Allocation and Interference Mitigation Schemes in D2D Enabled Networks
5th generation networks are envisioned to provide seamless and ubiquitous connection to 1000-fold more devices and is believed to provide ultra-low latency and higher data rates up to tens of Gbps. Different technologies enabling these requirements are being developed including mmWave communications, Massive MIMO and beamforming, Device to Device (D2D) communications and Heterogeneous Networks. D2D communication is a promising technology to enable applications requiring high bandwidth such as online streaming and online gaming etc. It can also provide ultra- low latencies required for applications like vehicle to vehicle communication for autonomous driving. D2D communication can provide higher data rates with high energy efficiency and spectral efficiency compared to conventional communication. The performance benefits of D2D communication can be best achieved when D2D users reuses the spectrum being utilized by the conventional cellular users. This spectrum sharing in a multi-tier heterogeneous network will introduce complex interference among D2D users and cellular users which needs to be resolved. Motivated by limited number of surveys for interference mitigation and resource allocation in D2D enabled heterogeneous networks, we have surveyed different conventional and artificial intelligence based interference mitigation and resource allocation schemes developed in recent years. Our contribution lies in the analysis of conventional interference mitigation techniques and their shortcomings. Finally, the strengths of AI based techniques are determined and open research challenges deduced from the recent research are presented.
Chargrid: Towards Understanding 2D Documents
We introduce a novel type of text representation that preserves the 2D layout of a document. This is achieved by encoding each document page as a two-dimensional grid of characters. Based on this representation, we present a generic document understanding pipeline for structured documents. This pipeline makes use of a fully convolutional encoder-decoder network that predicts a segmentation mask and bounding boxes. We demonstrate its capabilities on an information extraction task from invoices and show that it significantly outperforms approaches based on sequential text or document images.
Vis-DSS: An Open-Source toolkit for Visual Data Selection and Summarization
Graph Pattern Polynomials
We study the time complexity of induced subgraph isomorphism problems where the pattern graph is fixed. The earliest known example of an improvement over trivial algorithms is by Itai and Rodeh (1978) who sped up triangle detection in graphs using fast matrix multiplication. This algorithm was generalized by Ne\v{s}et\v{r}il and Poljak (1985) to speed up detection of k-cliques. Improved algorithms are known for certain small-sized patterns. For example, a linear-time algorithm is known for detecting length-4 paths. In this paper, we give the first pattern detection algorithm that improves upon Ne\v{s}et\v{r}il and Poljak’s algorithm for arbitrarily large pattern graphs (not cliques). The algorithm is obtained by reducing the induced subgraph isomorphism problem to the problem of detecting multilinear terms in constant-degree polynomials. We show that the same technique can be used to reduce the induced subgraph isomorphism problem of many pattern graphs to constructing arithmetic circuits computing homomorphism polynomials of these pattern graphs. Using this, we obtain faster combinatorial algorithms (algorithms that do not use fast matrix multiplication) for k-paths and k-cycles. We also obtain faster algorithms for 5-paths and 5-cycles that match the runtime for triangle detection. We show that these algorithms are expressible using polynomial families that we call graph pattern polynomial families. We then define a notion of reduction among these polynomials that allows us to compare the complexity of various pattern detection problems within this framework. For example, we show that the induced subgraph isomorphism polynomial for any pattern that contains a k-clique is harder than the induced subgraph isomorphism polynomial for k-clique. An analogue of this theorem is not known with respect to general algorithmic hardness.
A Comparative Study: Adaptive Fuzzy Inference Systems for Energy Prediction in Urban Buildings
This investigation aims to study different adaptive fuzzy inference algorithms capable of real-time sequential learning and prediction of time-series data. A brief qualitative description of these algorithms namely meta-cognitive fuzzy inference system (McFIS), sequential adaptive fuzzy inference system (SAFIS) and evolving Takagi-Sugeno (ETS) model provide a comprehensive comparison of their working principle, especially their unique characteristics are discussed. These algorithms are then simulated with dataset collected at one of the academic buildings at Nanyang Technological University, Singapore. The performance are compared by means of the root mean squared error (RMSE) and non-destructive error index (NDEI) of the predicted output. Analysis shows that McFIS shows promising results either with lower RMSE and NDEI or with lower architectural complexity over ETS and SAFIS. Statistical Analysis also reveals the significance of the outcome of these algorithms.
Characterization of Biologically Relevant Network Structures form Time-series Data
High-throughput data acquisition in synthetic biology leads to an abundance of data that need to be processed and aggregated into useful biological models. Building dynamical models based on this wealth of data is of paramount importance to understand and optimize designs of synthetic biology constructs. However, building models manually for each data set is inconvenient and might become infeasible for highly complex synthetic systems. In this paper, we present state-of-the-art system identification techniques and combine them with chemical reaction network theory (CRNT) to generate dynamic models automatically. On the system identification side, Sparse Bayesian Learning offers methods to learn from data the sparsest set of dictionary functions necessary to capture the dynamics of the system into ODE models; on the CRNT side, building on such sparse ODE models, all possible network structures within a given parameter uncertainty region can be computed. Additionally, the system identification process can be complemented with constraints on the parameters to, for example, enforce stability or non-negativity—thus offering relevant physical constraints over the possible network structures. In this way, the wealth of data can be translated into biologically relevant network structures, which then steers the data acquisition, thereby providing a vital step for closed-loop system identification.
Deep Confidence: A Computationally Efficient Framework for Calculating Reliable Errors for Deep Neural Networks
Deep learning architectures have proved versatile in a number of drug discovery applications, including the modelling of in vitro compound activity. While controlling for prediction confidence is essential to increase the trust, interpretability and usefulness of virtual screening models in drug discovery, techniques to estimate the reliability of the predictions generated with deep learning networks remain largely underexplored. Here, we present Deep Confidence, a framework to compute valid and efficient confidence intervals for individual predictions using the deep learning technique Snapshot Ensembling and conformal prediction. Specifically, Deep Confidence generates an ensemble of deep neural networks by recording the network parameters throughout the local minima visited during the optimization phase of a single neural network. This approach serves to derive a set of base learners (i.e., snapshots) with comparable predictive power on average, that will however generate slightly different predictions for a given instance. The variability across base learners and the validation residuals are in turn harnessed to compute confidence intervals using the conformal prediction framework. Using a set of 24 diverse IC50 data sets from ChEMBL 23, we show that Snapshot Ensembles perform on par with Random Forest (RF) and ensembles of independently trained deep neural networks. In addition, we find that the confidence regions predicted using the Deep Confidence framework span a narrower set of values. Overall, Deep Confidence represents a highly versatile error prediction framework that can be applied to any deep learning-based application at no extra computational cost.
Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation
Event extraction is of practical utility in natural language processing. In the real world, it is a common phenomenon that multiple events existing in the same sentence, where extracting them are more difficult than extracting a single event. Previous works on modeling the associations between events by sequential modeling methods suffer a lot from the low efficiency in capturing very long-range dependencies. In this paper, we propose a novel Jointly Multiple Events Extraction (JMEE) framework to jointly extract multiple event triggers and arguments by introducing syntactic shortcut arcs to enhance information flow and attention-based graph convolution networks to model graph information. The experiment results demonstrate that our proposed framework achieves competitive results compared with state-of-the-art methods.
Autonomous Deep Learning: Incremental Learning of Denoising Autoencoder for Evolving Data Streams
The generative learning phase of Autoencoder (AE) and its successor Denosing Autoencoder (DAE) enhances the flexibility of data stream method in exploiting unlabelled samples. Nonetheless, the feasibility of DAE for data stream analytic deserves in-depth study because it characterizes a fixed network capacity which cannot adapt to rapidly changing environments. An automated construction of a denoising autoeconder, namely deep evolving denoising autoencoder (DEVDAN), is proposed in this paper. DEVDAN features an open structure both in the generative phase and in the discriminative phase where input features can be automatically added and discarded on the fly. A network significance (NS) method is formulated in this paper and is derived from the bias-variance concept. This method is capable of estimating the statistical contribution of the network structure and its hidden units which precursors an ideal state to add or prune input features. Furthermore, DEVDAN is free of the problem- specific threshold and works fully in the single-pass learning fashion. The efficacy of DEVDAN is numerically validated using nine non-stationary data stream problems simulated under the prequential test-then-train protocol where DEVDAN is capable of delivering an improvement of classification accuracy to recently published online learning works while having flexibility in the automatic extraction of robust input features and in adapting to rapidly changing environments.
Implicit Maximum Likelihood Estimation
Implicit probabilistic models are models defined naturally in terms of a sampling procedure and often induces a likelihood function that cannot be expressed explicitly. We develop a simple method for estimating parameters in implicit models that does not require knowledge of the form of the likelihood function or any derived quantities, but can be shown to be equivalent to maximizing likelihood under some conditions. Our result holds in the non-asymptotic parametric setting, where both the capacity of the model and the number of data examples are finite. We also demonstrate encouraging experimental results.
• Exponential Convergence Time of Gradient Descent for One-Dimensional Deep Linear Neural Networks• Neural Arithmetic Expression Calculator• Understanding the Gist of Images – Ranking of Concepts for Multimedia Indexing• A counterexample to a conjecture of Wang and Hou on signed graphs• Magic arrays with empty cells• Unit Ball Graphs on Geodesic Spaces• Nearly hyperharmonic functions are infima of excessive functions• Detecting Features of Tools, Objects, and Actions from Effects in a Robot using Deep Learning• Local mild solutions for rough stochastic partial differential equations• Accelerate CU Partition in HEVC using Large-Scale Convolutional Neural Network• Learning and Evaluating Sparse Interpretable Sentence Embeddings• Hierarchical Distributed Voltage Regulation in Networked Autonomous Grids• Unsupervised Learning of Dense Optical Flow and Depth from Sparse Event Data• Domain Adaptation in Robot Fault Diagnostic Systems• OS Scheduling Algorithms for Memory Intensive Workloads in Multi-socket Multi-core servers• Rainbow Ramsey problems for the Boolean lattice• Bounds on the nonnegative signed domination number of graphs• Analysis of Daily Streamflow Complexity by Kolmogorov Measures and Lyapunov Exponent• Classification theorem for strong triangle blocking arrangements• A Control Framework for Optimal Coordination and Desired Formation of Autonomous Agents• Path integral approach to analytic continuation of Liouville theory: the pencil region• Mind Your Language: Abuse and Offense Detection for Code-Switched Languages• Kanade–Russell conjectures and linked partition ideals• Accelerated Gossip via Stochastic Heavy Ball Method• Multi-View Community Detection in Facebook Public Pages• On the Information in Extreme Measurements for Parameter Estimation• Collapsing Superstring Conjecture• Curvilinear Structure Enhancement by Multiscale Top-Hat Tensor in 2D/3D Images• On the quenched CLT for stationary random fields under projective criteria• Boxes, extended boxes, and sets of positive upper density in the Euclidean space• Second-order Guarantees of Distributed Gradient Algorithms• Quantitatively Admissible Representations and the ‘Main Theorem’ of Type-2 COMPLEXITY Theory• A Learning Theory Approach to a Computationally Efficient Parameter Selection for the Elastic Net• Textually Enriched Neural Module Networks for Visual Question Answering• Distance problems and extension theorems over finite fields• Contribution of social network analysis and collective phenomena to understanding social complexity and cognition• On the Behavior of the Expectation-Maximization Algorithm for Mixture Models• Is Ordered Weighted $\ell_1$ Regularized Regression Robust to Adversarial Perturbation? A Case Study on OSCAR• Application of Machine Learning in Wireless Networks: Key Techniques and Open Issues• A Canonical Form for First-Order Distributed Optimization Algorithms• Recognizing Film Entities in Podcasts• Decentralized Stochastic Control with a Word-of-Mouth Communication• Deep Knowledge Tracing and Dynamic Student Classification for Knowledge Tracing• Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback• Unified recurrent neural network for many feature types• Synchronization of Kuramoto Oscillators: Inverse Taylor Expansions• The Combinatorial Data Fusion Problem in Conflicted-supervised Learning• Context-Aware Attention for Understanding Twitter Abuse• Fluctuations of dimer heights on contracting square-hexagon lattices• Sentence-Level Fluency Evaluation: References Help, But Can Be Spared!• Penalized regression adjusted causal effect estimates in high dimensional randomized experiments• Neural Transductive Learning and Beyond: Morphological Generation in the Minimal-Resource Setting• Modern Convex Optimization to Medical Image Analysis• Streaming dynamic and distributed inference of latent geometric structures• Unified Necessary and Sufficient Conditions for the Robust Stability of Interconnected Sector-Bounded Systems• Interactions as Social Practices: towards a formalization• Learning to Detect Fake Face Images in the Wild• Multi-part cross-intersecting families• Low Frequency Adversarial Perturbation• Moment bounds for autocovariance matrices under dependence• Speaker Naming in Movies• Grassmann graphs, degenerate DAHA, and non-symmetric dual $q$-Hahn polynomials• Graphical zonotopes with the same face vector• FCHD: A fast and accurate head detector• Optimal Multicast of Tiled 360 VR Video• Longitudinal data analysis using matrix completion• Norm-Ranging LSH for Maximum Inner Product Search• Person Identification using Seismic Signals generated from Footfalls• Analytical Derivation of the Impulse Response for the Bounded 2-D Diffusion Channel• Modeling non-linear spectral domain dependence using copulas with applications to rat local field potentials• Full-Duplex Transmission Optimization for Bi-directional MIMO links with QoS Guarantees• In the Shadows of a hypergraph: looking for associated primes of powers of squarefree monomial ideals• Beyond Binomial and Negative Binomial: Adaptation in Bernoulli Parameter Estimation• MobileFace: 3D Face Reconstruction with Efficient CNN Regression• Convergence rates for Penalised Least Squares Estimators in PDE-constrained regression problems• Oscillation Damping Control of Pendulum-like Manipulation Platform using Moving Masses• Orthogonally Decoupled Variational Gaussian Processes• Almost optimal algorithms for diameter-optimally augmenting trees• Representing Sets as Summed Semantic Vectors• Information-Weighted Neural Cache Language Models for ASR• Wasserstein Distributionally Robust Kalman Filtering• Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning• Dense neural networks as sparse graphs and the lightning initialization• ROI constrained Auctions• An interior point sequential quadratic programming-type method for log-determinant semi-infinite programs• Inference of the Kinetic Ising Model with Heterogeneous Missing Data• Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Function• A Framework towards Domain Specific Video Summarization• A limit theorem for the survival probability of a simple random walk among power-law renewal traps• Semi Parametric Estimations of rotating and scaling parameters for aeronautic loads• Preserving the distribution function in surveys in case of imputation for zero inflated data• Classify, predict, detect, anticipate and synthesize: Hierarchical recurrent latent variable models for human activity modeling• Stochastic stability of a system of perfect integrate-and-fire inhibitory neurons• Vision-based Control of a Quadrotor in User Proximity: Mediated vs End-to-End Learning Approaches• Robotics Rights and Ethics Rules• Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task• An Automated Approach Towards Sparse Single-Equation Cointegration Modelling• On the Simpson index for the Moran process with random selection and immigration• Edge metric dimension of some graph operations• Fast, Precise Myelin Water Quantification using DESS MRI and Kernel Learning• Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings• Hype Cycle Dynamics: Microscopic Modeling and Detection• Dissipative solitons for bistable delayed-feedback systems• Galois extensions, positive involutions and an application to unitary space-time coding• Sampling by Intersections with Random Geodesics• WiRe57 : A Fine-Grained Benchmark for Open Information Extraction• On local Dressians of matroids• A Bayesian Approach to Multivariate Adaptive Localization in Ensemble-Based Data Assimilation with Time-Dependent Extensions• On The Utility of Conditional Generation Based Mutual Information for Characterizing Adversarial Subspaces• Sufficient condition for Reed’s conjecture• Analysis and optimisation of a variational model for mixed Gaussian and Salt & Pepper noise removal• Improved Semantic Stixels via Multimodal Sensor Fusion• Fundamental Limits of Spectrum Sharing for NOMA-based Cooperative Relaying• Fast Geometrically-Perturbed Adversarial Faces• A short note on ‘Anticipative portfolio optimization’• SDN Flow Entry Management Using Reinforcement Learning• Weakly-Supervised Learning of Metric Aggregations for Deformable Image Registration• Natural Gas Flow Equations: Uniqueness and an MI-SOCP Solver• On quadratic optimization problems and canonical duality theory• Combinatorial interpretations of Lucas analogues of binomial coefficients and Catalan numbers• Optimal Distributed Control of a Cahn-Hilliard-Darcy System with Mass Sources• Extensions of Generalized Two-Qubit Separability Probability Analyses to Higher Dimensions, Additional Measures and New Methodologies• Localization near the edge for the Anderson Bernoulli model on the two dimensional lattice• A comparative tour through the simulation algorithms for max-stable processes• An experimental approach for global polynomial optimization based on Moments and Semidefinite Programming• Long Time Behavior of First Order Mean Field Games on Euclidean Space• Sparse-to-Continuous: Enhancing Monocular Depth Estimation using Occupancy Maps• Optimality of Linear Sketching under Modular Updates• An extension of Heston’s SV model to Stochastic Interest Rates• Incorporating Luminance, Depth and Color Information by Fusion-based Networks for Semantic Segmentation• Smoothing of Boundary Behaviour in Stochastic Planar Evolutions
Like this:
Like Loading…
Related