Use this to configure a SparkContext for a your task. Task’s marked with this will automatically execute
natively onto K8s as a distributed execution of spark
Parameters
spark_conf (Optional[Dict[str, str]]) – Dictionary of spark config. The variables should match what spark expects
hadoop_conf (Optional[Dict[str, str]]) – Dictionary of hadoop conf. The variables should match a typical hadoop configuration for spark
