Syntax for UPSERT using a subquery

Hi all, I can't seem to figure this one out so maybe someone has hit the same issue. I am guessing it's a syntax problem.

UPSERT works fine when dealing with a single row, but I can't get this to work for a subquery.

Works fine

UPSERT <TABLE> (COLUMN1, COLUMN2) VALUES ('VALUE1', 'VALUE2') WITH PRIMARY KEY;

Syntax error near "PRIMARY"

UPSERT <TABLE> (COLUMN1, COLUMN2) VALUES (SELECT COLUMN1, COLUMN2 FROM TABLE2) WITH PRIMARY KEY;

Same thing tried another way

UPSERT <TABLE> (COLUMN1, COLUMN2) (SELECT COLUMN1, COLUMN2 FROM TABLE2) WITH PRIMARY KEY;

I have used this successfully before with INSERT, but I am thinking the UPSERT is a slightly different animal.

Regards,

Justin