You can delete a document(s) attached to a task by opening the task detail page (from either the Tasks or Matters module). In the Documents section, click the trash icon for an individual document, or the 'Clear' button for all documents attached to the task.

Please Note: This does not delete the document itself. It only removes the link between the file and the task. The document still exists in the same folder and can be reattached if necessary.