I will present the results on the evolution of dust particles in self-gravitating disks residing in a gravitoturbulent state, when heating due to shocks of density waves balances cooling. It is well known that density structures in the gaseous component of the disk induced by self-gravity (gravitational instability) can trap dust efficiently enough, so that the dust component itself undergoes further gravitational collapse due to its own self-gravity. Previous results both in global and local shearing box studies indicate that over-pressure regions related to spiral density waves can be very efficient at collecting dust particles, creating significant local over-densities of particles. The degree of such concentrations depends on two parameters: the size of dust particles and the rate of gas cooling. In recent years, increasing observational evidence indicates that large-scale vortices (e.g., induced by planetary gaps) and rings are most preferable sites of dust trapping.
Motivated by this, we studied effects of vortices on the evolution of dust particles using local shearing box simulations of self-gravitating protoplanetary disks, including also the dust-back reaction on gas and self-gravity of the dust component itself. In contrast to non-self-gravitating disks, vortices in self-gravitating disks tend to be smaller-scale (of the order of local Jeans scale) and short-lived structures. We found that these types of structures are nevertheless quite efficient at trapping small and intermediate-sized dust particles with friction times comparable to, or less than, the local orbital period of the disk. This can lead to significant over-densities in the solid component of the disk, with density enhancements comparable to, and even higher, than those within spiral density waves; increasing the rate of gravitational collapse of dust into bound structures (planetesimals). I will also discuss the resulting surface density structure of dust trapped in such vortices in connection with recent observations of disks.