The transaction is expected to close in late 2020 or early 2021.
The property, located in Bossier City, includes a horse racing track and racino with 1,300 slot machines.
Caesars purchased the property in 2002, then transferred it to VICI in October 2017 as part of a spin-off.
The proceeds will be split $16.5m to Caesars and $5.5m to VICI.
Louisiana Downs, which opened in 1974, closed for two months amidst the coronavirus pandemic before reopening the casino on 18 May.