Find out which processes is using your swap
To find out which process is using your swap memory, use smem
$ sudo apt-get install smem
# To get a full list of PID and the amount of memory and swap they use
$ smem -s swap
# To get a summary of memory usage by user
$ smem -u
References: