You will need to have Project Admin, Account Admin, or Account Owner permissions level in order to delete a download.
In the Downloads tab, click on "...More" > "Delete Download".
Once deleted, you will no longer have access to the deleted Download. The Download tag/document identifiers associated with the Download will also be removed. You will receive a Download deletion email like this:
If this was a mistake, you can always re-create the Download.