We develop methodology and theory for a general Bayesian approach towards dynamic variable selection in high dimensional regression models with time varying parameters. Specifically, we propose a variational inference scheme which features dynamic sparsity inducing properties so that different subsets of “active” predictors can be identified over different time periods. We compare our modeling framework against established static and dynamic variable selection methods both in simulation and within the context of two common problems in macroeconomics and finance, such as inflation forecasting and equity returns predictability. The results show that our approach helps to tease out more accurately the dynamic impact of different predictors over time. This translates into significant gains in terms of out of sample point and density forecasting accuracy. We believe our results highlight the importance of taking a dynamic approach towards variable selection for economic modeling and forecasting.