Hive - How to know which execution engine I am currently using
The Hive execution engine is controlled by hive.execution.engine
property. It can be either of the following:
-
mr
(Map Reduce, default) -
tez
(Tez execution, for Hadoop 2 only) -
spark
(Spark execution, for Hive 1.1.0 onward).
The property can be read & updated using hive/beeline cli
- For reading -
SET hive.execution.engine;
- For updating -
SET hive.execution.engine=tez;
If you want to programmatically get this value out, you must go for HiveClient which supports multiple ways like JDBC, Java, Python, PHP, Ruby, C++, etc.
References
- https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=82903061#ConfigurationProperties-hive.execution.engine
- https://cwiki.apache.org/confluence/display/Hive/HiveClient
Indrajeet Gour
Here for knowing things better then mine and also spreading my ideas and knowledge which could make more scene to others.
Updated on June 13, 2022Comments
-
Indrajeet Gour almost 2 years
I want to automate my hive ETL workflow in such a way that I need to execute hive jobs on the basis of execution engine (Tez or MR) because of memory constraints.
Would you please help, as I wanted to cross-check in-between of my whole work-flow which execution engine currently I'm dealing with.
Thanks in advance.