If you need to convert a TEXT column to a BYTEA column, you can use a cast expression to do the work:

ALTER TABLE mytable ALTER COLUMN col TYPE bytea USING
  decode(replace(col, '\\', '\\\\'), 'escape');

Note that altering the column type of a column is only supported beginning with PostgreSQL version 8.0. For more information, see the manual.