forked from apache/hive
/
RELEASE_NOTES.txt
2156 lines (2095 loc) · 164 KB
/
RELEASE_NOTES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Release Notes - Hive - Version 3.1.0
** Sub-task
* [HIVE-12192] - Hive should carry out timestamp computations in UTC
* [HIVE-17227] - Incremental replication load should create tasks in execution phase rather than semantic phase
* [HIVE-17657] - export/import for MM tables is broken
* [HIVE-18193] - Migrate existing ACID tables to use write id per table rather than global transaction id
* [HIVE-18748] - Rename table impacts the ACID behavior as table names are not updated in meta-tables.
* [HIVE-18840] - CachedStore: Prioritize loading of recently accessed tables during prewarm
* [HIVE-18946] - Fix columnstats merge NPE
* [HIVE-18988] - Support bootstrap replication of ACID tables
* [HIVE-19009] - Retain and use runtime statistics during hs2 lifetime
* [HIVE-19096] - query result cache interferes with explain analyze
* [HIVE-19126] - CachedStore: Use memory estimation to limit cache size during prewarm
* [HIVE-19128] - Update golden files for spark perf tests
* [HIVE-19135] - Need tool to allow admins to create catalogs and move existing dbs to catalog during upgrade
* [HIVE-19141] - TestNegativeCliDriver insert_into_notnull_constraint, insert_into_acid_notnull failing
* [HIVE-19159] - TestMTQueries.testMTQueries1 failure
* [HIVE-19164] - TestMetastoreVersion failures
* [HIVE-19171] - Persist runtime statistics in metastore
* [HIVE-19193] - TestActivePassiveHA fails
* [HIVE-19194] - TestDruidStorageHandler fails
* [HIVE-19195] - Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.
* [HIVE-19196] - TestTriggersMoveWorkloadManager is flaky
* [HIVE-19206] - Automatic memory management for open streaming writers
* [HIVE-19209] - Streaming ingest record writers should accept input stream
* [HIVE-19210] - Create separate module for streaming ingest
* [HIVE-19211] - New streaming ingest API and support for dynamic partitioning
* [HIVE-19214] - High throughput ingest ORC format
* [HIVE-19222] - TestNegativeCliDriver tests are failing due to "java.lang.OutOfMemoryError: GC overhead limit exceeded"
* [HIVE-19243] - Upgrade hadoop.version to 3.1.0
* [HIVE-19252] - TestJdbcWithMiniKdcCookie.testCookieNegative is failing consistently
* [HIVE-19274] - Add an OpTreeSignature persistence checker hook
* [HIVE-19332] - Disable compute.query.using.stats for external table
* [HIVE-19336] - Disable SMB/Bucketmap join for external tables
* [HIVE-19340] - Disable timeout of transactions opened by replication task at target cluster
* [HIVE-19347] - TestTriggersWorkloadManager tests are failing consistently
* [HIVE-19348] - org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp are failing
* [HIVE-19366] - Vectorization causing TestStreaming.testStreamBucketingMatchesRegularBucketing to fail
* [HIVE-19374] - Parse and process ALTER TABLE SET OWNER command syntax
* [HIVE-19409] - Disable incremental rewriting with outdated materialized views
* [HIVE-19472] - HiveStreamingConnection swallows exception on partition creation
* [HIVE-19494] - Accept shade prefix during reflective instantiation of output format
* [HIVE-19495] - Arrow SerDe itest failure
* [HIVE-19499] - Bootstrap REPL LOAD shall add tasks to create checkpoints for db/tables/partitions.
* [HIVE-19500] - Prevent multiple selectivity estimations for the same variable in conjuctions
* [HIVE-19562] - Flaky test: TestMiniSparkOnYarn FileNotFoundException in spark-submit
* [HIVE-19598] - Add Acid V1 to V2 upgrade module
* [HIVE-19637] - Add slow test report script to testutils
* [HIVE-19688] - Make catalogs updatable
* [HIVE-19727] - Fix Signature matching of table aliases
* [HIVE-19739] - Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded data/metadata.
* [HIVE-19758] - Set hadoop.version=3.1.0 in standalone-metastore
* [HIVE-19768] - Utility to convert tables to conform to Hive strict managed tables mode
* [HIVE-19799] - remove jasper dependency
* [HIVE-19815] - Repl dump should not propagate the checkpoint and repl source properties
* [HIVE-19851] - upgrade jQuery version
* [HIVE-19852] - update jackson to latest
* [HIVE-19868] - Add support for float aggregator
* [HIVE-19892] - Disable query results cache for for HiveServer2 doAs=true
* [HIVE-19923] - Follow up of HIVE-19615, use UnaryFunction instead of prefix
** Bug
* [HIVE-15190] - Field names are not preserved in ORC files written with ACID
* [HIVE-18434] - Type is not determined correctly for comparison between decimal column and string constant
* [HIVE-18816] - CREATE TABLE (ACID) doesn't work with TIMESTAMPLOCALTZ column type
* [HIVE-19016] - Vectorization and Parquet: Disable vectorization for nested complex types
* [HIVE-19054] - Function replication shall use "hive.repl.replica.functions.root.dir" as root
* [HIVE-19108] - Vectorization and Parquet: Turning on vectorization in parquet_ppd_decimal.q causes Wrong Query Results
* [HIVE-19109] - Vectorization: Enabling vectorization causes TestCliDriver delete_orig_table.q to produce Wrong Results
* [HIVE-19110] - Vectorization: Enabling vectorization causes TestContribCliDriver udf_example_arraymapstruct.q to produce Wrong Results
* [HIVE-19118] - Vectorization: Turning on vectorization in escape_crlf produces wrong results
* [HIVE-19120] - catalog not properly set for some tables in SQL upgrade scripts
* [HIVE-19131] - DecimalColumnStatsMergerTest comparison review
* [HIVE-19155] - Day time saving cause Druid inserts to fail with org.apache.hive.druid.io.druid.java.util.common.UOE: Cannot add overlapping segments
* [HIVE-19157] - Assert that Insert into Druid Table fails if the publishing of metadata by HS2 fails
* [HIVE-19167] - Map data type doesn't keep the order of the key/values pairs as read (Part 2, The Sequel or SQL)
* [HIVE-19168] - Ranger changes for llap commands
* [HIVE-19186] - Multi Table INSERT statements query has a flaw for partitioned table when INSERT INTO and INSERT OVERWRITE are used
* [HIVE-19200] - Vectorization: Disable vectorization for LLAP I/O when a non-VECTORIZED_INPUT_FILE_FORMAT mode is needed (i.e. rows) and data type conversion is needed
* [HIVE-19219] - Incremental REPL DUMP should throw error if requested events are cleaned-up.
* [HIVE-19230] - Schema column width inconsistency in Oracle
* [HIVE-19231] - Beeline generates garbled output when using UnsupportedTerminal
* [HIVE-19237] - Only use an operatorId once in a plan
* [HIVE-19247] - StatsOptimizer: Missing stats fast-path for Date
* [HIVE-19248] - REPL LOAD couldn't copy file from source CM path and also doesn't throw error if file copy fails.
* [HIVE-19258] - add originals support to MM tables (and make the conversion a metadata only operation)
* [HIVE-19264] - Vectorization: Reenable vectorization in vector_adaptor_usage_mode.q
* [HIVE-19269] - Vectorization: Turn On by Default
* [HIVE-19275] - Vectorization: Defer Wrong Results / Execution Failures when Vectorization turned on
* [HIVE-19277] - Active/Passive HA web endpoints does not allow cross origin requests
* [HIVE-19312] - MM tables don't work with BucketizedHIF
* [HIVE-19317] - Handle schema evolution from int like types to decimal
* [HIVE-19327] - qroupby_rollup_empty.q fails for insert-only transactional tables
* [HIVE-19331] - Repl load config in "with" clause not pass to Context.getStagingDir
* [HIVE-19350] - Vectorization: Turn off vectorization for explainuser_1.q / spark_explainuser_1
* [HIVE-19352] - Vectorization: Disable vectorization for org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData
* [HIVE-19357] - Vectorization: assert_true HiveException erroneously gets suppressed to NULL
* [HIVE-19358] - CBO decorrelation logic should generate Hive operators
* [HIVE-19365] - Index on COMPLETED_TXN_COMPONENTS in Metastore RDBMS has different names in different scripts
* [HIVE-19370] - Issue: ADD Months function on timestamp datatype fields in hive
* [HIVE-19381] - Function replication in cloud fail when download resource from AWS
* [HIVE-19382] - Acquire locks before generating valid transaction list for some operations
* [HIVE-19384] - Vectorization: IfExprTimestamp* do not handle NULLs correctly
* [HIVE-19385] - Optional hive env variable to redirect bin/hive to use Beeline
* [HIVE-19389] - Schematool: For Hive's Information Schema, use embedded HS2 as default
* [HIVE-19410] - don't create serde reader in LLAP if there's no cache
* [HIVE-19418] - add background stats updater similar to compactor
* [HIVE-19423] - REPL LOAD creates staging directory in source dump directory instead of table data location
* [HIVE-19433] - HiveJoinPushTransitivePredicatesRule hangs
* [HIVE-19435] - Incremental replication cause data loss if a table is dropped followed by create and insert-into with different partition type.
* [HIVE-19454] - Test failure : org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails with java.lang.AssertionError
* [HIVE-19460] - Improve stats estimations for NOT IN operator
* [HIVE-19463] - TezTask - getting groups may fail (PartialGroupNameException in some tests)
* [HIVE-19467] - Make storage format configurable for temp tables created using LLAP external client
* [HIVE-19474] - Decimal type should be casted as part of the CTAS or INSERT Clause.
* [HIVE-19479] - encoded stream seek is incorrect for 0-length RGs in LLAP IO
* [HIVE-19481] - Tablesample uses incorrect logic to pick files corresponding to buckets.
* [HIVE-19485] - dump directory for non native tables should not be created
* [HIVE-19493] - VectorUDFDateDiffColCol copySelected does not handle nulls correctly
* [HIVE-19496] - Check untar folder
* [HIVE-19498] - Vectorization: CAST expressions produce wrong results
* [HIVE-19504] - Change default value for hive.auto.convert.join.shuffle.max.size property
* [HIVE-19516] - TestNegative merge_negative_5 and mm_concatenate are causing timeouts
* [HIVE-19529] - Vectorization: Date/Timestamp NULL issues
* [HIVE-19557] - stats: filters for dates are not taking advantage of min/max values
* [HIVE-19565] - Vectorization: Fix NULL / Wrong Results issues in STRING Functions
* [HIVE-19567] - Fix flakiness in TestTriggers
* [HIVE-19569] - alter table db1.t1 rename db2.t2 generates MetaStoreEventListener.onDropTable()
* [HIVE-19575] - TestAutoPurgeTables seems flaky
* [HIVE-19577] - CREATE TEMPORARY TABLE LIKE and INSERT generate output format mismatch errors
* [HIVE-19578] - HLL merges tempList on every add
* [HIVE-19588] - Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
* [HIVE-19589] - Disable TestAutoPurge tests and annotate TestTriggersWorkloadManager with retry
* [HIVE-19590] - mask stats in llap_smb
* [HIVE-19592] - TestWorkloadManager - add retry for now
* [HIVE-19594] - Add custom tmp folders to tests to avoid collisions
* [HIVE-19595] - Regenerate webui port in MiniHS2
* [HIVE-19604] - Incorrect Handling of Boolean in DruidSerde
* [HIVE-19605] - TAB_COL_STATS table has no index on db/table name
* [HIVE-19608] - disable flaky tests 2
* [HIVE-19613] - GenericUDTFGetSplits should handle fetch task with temp table rewrite
* [HIVE-19614] - GenericUDTFGetSplits does not honor ORDER BY
* [HIVE-19615] - Proper handling of is null and not is null predicate when pushed to Druid
* [HIVE-19619] - Allow comparisons between doubles and bigints
* [HIVE-19629] - Enable Decimal64 reader after orc version upgrade
* [HIVE-19631] - reduce epic locking in AbstractService
* [HIVE-19632] - Remove webapps directory from standalone jar
* [HIVE-19639] - a transactional Hive table cannot be imported as an external table
* [HIVE-19643] - MM table conversion doesn't need full ACID structure checks
* [HIVE-19644] - change WM syntax to avoid conflicts with identifiers starting with a number
* [HIVE-19646] - Filesystem closed error in HiveProtoLoggingHook
* [HIVE-19660] - update branch-3 to be version 3.1 and fix storage-api mismatch
* [HIVE-19675] - Cast to timestamps on Druid time column leads to an exception
* [HIVE-19677] - Disable sample6.q
* [HIVE-19680] - Push down limit is not applied for Druid storage handler.
* [HIVE-19684] - Hive stats optimizer wrongly uses stats against non native tables
* [HIVE-19687] - Export table on acid partitioned table is failing
* [HIVE-19690] - multi-insert query with multiple GBY, and distinct in only some branches can produce incorrect results
* [HIVE-19691] - Start SessionState in materialized views registry
* [HIVE-19695] - Year Month Day extraction functions need to add an implicit cast for column that are String types
* [HIVE-19697] - TestReOptimization#testStatCachingMetaStore is flaky
* [HIVE-19698] - TestAMReporter#testMultipleAM is flaky
* [HIVE-19700] - Workaround for JLine issue with UnsupportedTerminal
* [HIVE-19713] - itests/hive-jmh should not reference a concreate storage-api version
* [HIVE-19723] - Arrow serde: "Unsupported data type: Timestamp(NANOSECOND, null)"
* [HIVE-19726] - ORC date PPD is broken
* [HIVE-19728] - beeline with USE_BEELINE_FOR_HIVE_CLI fails when trying to set hive.aux.jars.path
* [HIVE-19734] - Beeline: When beeline-site.xml is present, beeline does not honor -n (username) and -p (password) arguments
* [HIVE-19744] - In Beeline if -u is specified the default connection should not be tried at all
* [HIVE-19750] - Initialize NEXT_WRITE_ID. NWI_NEXT on converting an existing table to full acid
* [HIVE-19753] - Strict managed tables mode in Hive
* [HIVE-19754] - vector_decimal_2 failing on branch-3
* [HIVE-19755] - insertsel_fail.q.out needs to be updated on branch-3
* [HIVE-19762] - Druid Queries containing Joins gives wrong results.
* [HIVE-19771] - allowNullColumnForMissingStats should not be false when column stats are estimated
* [HIVE-19772] - Streaming ingest V2 API can generate invalid orc file if interrupted
* [HIVE-19773] - CBO exception while running queries with tables that are not present in materialized views
* [HIVE-19777] - NPE in TezSessionState
* [HIVE-19789] - reenable orc_llap test
* [HIVE-19793] - disable LLAP IO batch-to-row wrapper for ACID deletes/updates
* [HIVE-19794] - Disable removing order by from subquery in GenericUDTFGetSplits
* [HIVE-19796] - Push Down TRUNC Fn to Druid Storage Handler
* [HIVE-19801] - JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes
* [HIVE-19808] - GenericUDTFGetSplits should support ACID reads in the temp. table read path
* [HIVE-19810] - StorageHandler fail to ship jars in Tez intermittently
* [HIVE-19813] - SessionState.start don't have to be synchronized
* [HIVE-19817] - Hive streaming API + dynamic partitioning + json/regex writer does not work
* [HIVE-19826] - OrcRawRecordMerger doesn't work for more than one file in non vectorized case
* [HIVE-19827] - hiveserver2 startup should provide a way to override TEZ_CONF_DIR
* [HIVE-19833] - reduce LLAP IO min allocation to match ORC variable CB size
* [HIVE-19837] - Setting to have different default location for external tables
* [HIVE-19838] - simplify & fix ColumnizedDeleteEventRegistry load loop
* [HIVE-19853] - Arrow serializer needs to create a TimeStampMicroTZVector instead of TimeStampMicroVector
* [HIVE-19857] - Set 3.1.0 for sys db version
* [HIVE-19859] - Inspect lock components for DBHiveLock while verifying whether transaction list is valid
* [HIVE-19861] - Fix temp table path generation for acid table export
* [HIVE-19862] - Postgres init script has a glitch around UNIQUE_DATABASE
* [HIVE-19864] - Address TestTriggersWorkloadManager flakiness
* [HIVE-19866] - improve LLAP cache purge
* [HIVE-19869] - Remove double formatting bug followup of HIVE-19382
* [HIVE-19872] - hive-schema-3.1.0.hive.sql is missing on master and branch-3
* [HIVE-19873] - Cleanup operation log on query cancellation after some delay
* [HIVE-19875] - increase LLAP IO queue size for perf
* [HIVE-19876] - Multiple fixes for Driver.isValidTxnListState
* [HIVE-19877] - Remove setting hive.execution.engine as mr in HiveStreamingConnection
* [HIVE-19879] - Remove unused calcite sql operator.
* [HIVE-19884] - Invalidation cache may throw NPE when there is no data in table used by materialized view
* [HIVE-19889] - Wrong results due to PPD of non deterministic functions with CBO
* [HIVE-19890] - ACID: Inherit bucket-id from original ROW_ID for delete deltas
* [HIVE-19898] - Disable TransactionalValidationListener when the table is not in the Hive catalog
* [HIVE-19903] - Disable temporary insert-only transactional table
* [HIVE-19904] - Load data rewrite into Tez job fails for ACID
* [HIVE-19908] - Block Insert Overwrite with Union All on full CRUD ACID tables using HIVE_UNION_SUBDIR_
* [HIVE-19912] - Schema evolution checks prints a log line in INFO mode for each vectorized rowbatch, impacts performance
* [HIVE-19917] - Export of full CRUD transactional table fails if table is not in default database
* [HIVE-19920] - Schematool fails in embedded mode when auth is on
* [HIVE-19921] - Fix perf duration and queue name in HiveProtoLoggingHook
* [HIVE-19938] - Upgrade scripts for information schema
* [HIVE-19941] - Row based Filters added via Hive Ranger policies are not pushed to druid
* [HIVE-19946] - VectorizedRowBatchCtx.recordIdColumnVector cannot be shared between different JVMs
* [HIVE-19951] - Vectorization: Need to disable encoded LLAP I/O for ORC when there is data type conversion (Schema Evolution)
* [HIVE-19956] - Include yarn registry classes to jdbc standalone jar
* [HIVE-19964] - Apply resource plan fails if trigger expression has quotes
* [HIVE-19965] - Make HiveEndPoint use IMetaStoreClient.add_partition
* [HIVE-19972] - Followup to HIVE-19928 : Fix the check for managed table
* [HIVE-19973] - Enable materialized view rewriting by default
* [HIVE-19980] - GenericUDTFGetSplits fails when order by query returns 0 rows
* [HIVE-19997] - Batches for TestMiniDruidCliDriver
* [HIVE-20002] - Shipping jdbd-storage-handler dependency jars in LLAP
* [HIVE-20010] - Fix create view over literals
* [HIVE-20076] - ACID: Fix Synthetic ROW__ID generation for vectorized orc readers
** New Feature
* [HIVE-18739] - Add support for Import/Export from Acid table
* [HIVE-19307] - Support ArrowOutputStream in LlapOutputFormatService
** Improvement
* [HIVE-17824] - msck repair table should drop the missing partitions from metastore
* [HIVE-18079] - Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator bit-size
* [HIVE-18394] - Materialized view: "Create Materialized View" should default to rewritable ones
* [HIVE-18410] - [Performance][Avro] Reading flat Avro tables is very expensive in Hive
* [HIVE-18743] - CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
* [HIVE-18792] - Allow standard compliant syntax for insert on partitioned tables
* [HIVE-18866] - Semijoin and analyze: Implement a Long -> Hash64 vector fast-path
* [HIVE-19027] - Make materializations invalidation cache work with multiple active remote metastores
* [HIVE-19161] - Add authorizations to information schema
* [HIVE-19228] - Remove commons-httpclient 3.x usage
* [HIVE-19259] - Create view on tables having union all fail with "Table not found"
* [HIVE-19344] - Change default value of msck.repair.batch.size
* [HIVE-19390] - Useless error messages logged for dummy table stats
* [HIVE-19415] - Support CORS for all HS2 web endpoints
* [HIVE-19421] - Upgrade version of Jetty to 9.3.20.v20170531
* [HIVE-19440] - Make StorageBasedAuthorizer work with information schema
* [HIVE-19462] - Fix mapping for char_length function to enable pushdown to Druid.
* [HIVE-19464] - Upgrade Parquet to 1.10.0
* [HIVE-19465] - Upgrade ORC to 1.5.0
* [HIVE-19466] - Update constraint violation error message
* [HIVE-19490] - Locking on Insert into for non native and managed tables.
* [HIVE-19534] - Allow implementations to access member variables of AbstractRecordWriter
* [HIVE-19560] - Retry test runner and retry rule for flaky tests
* [HIVE-19572] - Add option to mask stats and data size in q files
* [HIVE-19586] - Optimize Count(distinct X) pushdown based on the storage capabilities
* [HIVE-19669] - Upgrade ORC to 1.5.1
* [HIVE-19682] - Provide option for GenericUDTFGetSplits to return only schema metadata
* [HIVE-19775] - Schematool should use HS2 embedded mode in privileged auth mode
* [HIVE-19776] - HiveServer2.startHiveServer2 retries of start has concurrency issues
* [HIVE-19824] - Improve online datasize estimations for MapJoins
* [HIVE-19885] - Druid Kafka Ingestion - Allow user to set kafka consumer properties via table properties
* [HIVE-20102] - Add a couple of additional tests for query parsing
* [HIVE-20135] - Fix incompatible change in TimestampColumnVector to default to UTC
** Test
* [HIVE-19271] - TestMiniLlapLocalCliDriver default_constraint and check_constraint failing
* [HIVE-19515] - TestRpc.testServerPort is consistently failing
* [HIVE-19555] - Enable TestMiniLlapLocalCliDriver#tez_dynpart_hashjoin_1.q and TestMiniLlapLocalCliDriver#tez_vector_dynpart_hashjoin_1.q
* [HIVE-19573] - Fix flaky TestMiniLlapLocalCliDriver#explainuser_4.q
* [HIVE-19612] - Add option to mask lineage in q files
* [HIVE-19617] - Rename test tables to avoid collisions during execution in batches
* [HIVE-19620] - Change tmp directory used by PigServer in HCat tests
* [HIVE-19626] - Change tmp staging mapred directory for CliDriver
* [HIVE-19654] - Change tmp staging mapred directory for TestBlobstoreCliDriver
* [HIVE-19655] - Mask stats for TestMiniLlapLocalCliDriver#smb_mapjoin_15
* [HIVE-19699] - Re-enable TestReOptimization
* [HIVE-19706] - Disable TestJdbcWithMiniHS2#testHttpRetryOnServerIdleTimeout
* [HIVE-19731] - Change staging tmp directory used by TestHCatLoaderComplexSchema
* [HIVE-20123] - Fix masking tests after HIVE-19617
** Task
* [HIVE-18875] - Enable SMB Join by default in Tez
* [HIVE-19134] - Update copyright NOTICE and fix rat check failures
* [HIVE-19140] - Update metastore upgrade scripts to prepare for 3.1.0 development
* [HIVE-19257] - HIVE-19157 commit references wrong jira
* [HIVE-19306] - Arrow batch serializer
* [HIVE-19308] - Provide an Arrow stream reader for external LLAP clients
* [HIVE-19323] - Create metastore SQL install and upgrade scripts for 3.1
* [HIVE-19488] - Enable CM root based on db parameter, identifying a db as source of replication.
* [HIVE-19509] - Disable tests that are failing continuously
* [HIVE-19512] - If parallel execution is enabled, metastore is throwing out of sequence error.
* [HIVE-19708] - Repl copy retrying with cm path even if the failure is due to network issue
* [HIVE-19725] - Add ability to dump non-native tables in replication metadata dump
* [HIVE-19880] - Repl Load to return recoverable vs non-recoverable error codes
* [HIVE-19881] - Allow metadata-only dump for database which are not source of replication
* [HIVE-19928] - Load Data for managed tables should set the owner of loaded files to a configurable user
* [HIVE-19978] - Backport HIVE-18037 to branch-3
Release Notes - Hive - Version 3.0.0
** Sub-task
* [HIVE-11133] - Support hive.explain.user for Spark
* [HIVE-11418] - Dropping a database in an encryption zone with CASCADE and trash enabled fails
* [HIVE-13567] - Enable auto-gather column stats by default
* [HIVE-13583] - E061-14: Search Conditions
* [HIVE-13673] - LLAP: handle case where no service instance is found on the host specified in the input split
* [HIVE-14412] - Add timestamp with time zone
* [HIVE-14487] - Add REBUILD statement for materialized views
* [HIVE-14495] - Add SHOW MATERIALIZED VIEWS statement
* [HIVE-14498] - Freshness period for query rewriting using materialized views
* [HIVE-14518] - Support 'having' translation for Druid GroupBy queries
* [HIVE-14747] - Remove JAVA paths from profiles by sending them from ptest-client
* [HIVE-14947] - Add support for Acid 2 in Merge
* [HIVE-15016] - Run tests with Hadoop 3.0.0-beta1
* [HIVE-15018] - ALTER rewriting flag in materialized view
* [HIVE-15051] - Test framework integration with findbugs, rat checks etc.
* [HIVE-15173] - Allow dec as an alias for decimal
* [HIVE-15212] - merge branch into master
* [HIVE-15326] - Hive shims report Unrecognized Hadoop major version number: 3.0.0-alpha2-SNAPSHOT
* [HIVE-15436] - Enhancing metastore APIs to retrieve only materialized views
* [HIVE-15490] - REPL LOAD & DUMP support for INSERT events with change management
* [HIVE-15619] - Column pruner should handle DruidQuery
* [HIVE-15642] - Replicate Insert Overwrites, Dynamic Partition Inserts and Loads
* [HIVE-15673] - Allow multiple queries with disjunction
* [HIVE-15705] - Event replication for constraints
* [HIVE-15725] - Make it possible to run checkstyle for a specific module
* [HIVE-15758] - Allow correlated scalar subqueries with aggregates which has non-equi join predicates
* [HIVE-15834] - Add unit tests for org.json usage on master
* [HIVE-15899] - Make CTAS with acid target table and insert into acid_tbl select ... union all ... work
* [HIVE-15939] - Make cast expressions comply more to sql2011
* [HIVE-15982] - Support the width_bucket function
* [HIVE-15986] - Support "is [not] distinct from"
* [HIVE-16171] - Support replication of truncate table
* [HIVE-16186] - REPL DUMP shows last event ID of the database even if we use LIMIT option.
* [HIVE-16197] - Incremental insert into a partitioned table doesn't get replicated.
* [HIVE-16207] - Add support for Complex Types in Fast SerDe
* [HIVE-16228] - Support subqueries in complex expression in SELECT clause
* [HIVE-16256] - Flaky test: TestCliDriver.testCliDriver[comments]
* [HIVE-16266] - Enable function metadata to be written during bootstrap
* [HIVE-16267] - Enable bootstrap function metadata to be loaded in repl load
* [HIVE-16268] - enable incremental repl dump to handle functions metadata
* [HIVE-16269] - enable incremental function dump to be loaded via repl load
* [HIVE-16272] - support for drop function in incremental replication
* [HIVE-16276] - Fix NoSuchMethodError: com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V
* [HIVE-16294] - Support snapshot for truncate table
* [HIVE-16312] - Flaky test: TestHCatClient.testTransportFailure
* [HIVE-16313] - Flaky test: TestBeeLineDriver[drop_with_concurrency]
* [HIVE-16320] - Flaky test: TestBeeLineDriver.testCliDriver[escape_comments]
* [HIVE-16330] - Improve plans for scalar subquery with aggregates
* [HIVE-16344] - Test and support replication of exchange partition
* [HIVE-16372] - Enable DDL statement for non-native tables (add/remove table properties)
* [HIVE-16400] - Fix the MDC reference to use slf4j rather than log4j
* [HIVE-16416] - Service: move constants out from HiveAuthFactory
* [HIVE-16467] - Flaky test: TestCliDriver.testCliDriver[vector_order_null]
* [HIVE-16488] - Support replicating into existing db if the db is empty
* [HIVE-16493] - Skip column stats when colStats is empty
* [HIVE-16504] - Addition of binary licenses broke rat check
* [HIVE-16530] - Add HS2 operation logs and improve logs for REPL commands
* [HIVE-16532] - HIVE on hadoop 3 build failed due to hdfs client/server jar separation
* [HIVE-16535] - Hive fails to build from source code tarball
* [HIVE-16542] - make merge that targets acid 2.0 table fail-fast
* [HIVE-16555] - Add a new thrift API call for get_metastore_uuid
* [HIVE-16556] - Modify schematool scripts to initialize and create METASTORE_DB_PROPERTIES table
* [HIVE-16566] - Set column stats default as true when creating new tables/partitions
* [HIVE-16568] - Support complex types in external LLAP InputFormat
* [HIVE-16579] - CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table
* [HIVE-16586] - Fix Unit test failures when CachedStore is enabled
* [HIVE-16591] - DR for function Binaries on HDFS
* [HIVE-16600] - Refactor SetSparkReducerParallelism#needSetParallelism to enable parallel order by in multi_insert cases
* [HIVE-16601] - Display Session Id and Query Name / Id in Spark UI
* [HIVE-16617] - Clean up javadoc from errors in module hive-shims
* [HIVE-16618] - Clean up javadoc from errors in module hive-common
* [HIVE-16619] - Clean up javadoc from errors in module hive-serde
* [HIVE-16628] - Fix query25 when it uses a mix of MergeJoin and MapJoin
* [HIVE-16637] - Improve end-of-data checking for LLAP input format
* [HIVE-16642] - New Events created as part of replv2 potentially break replv1
* [HIVE-16644] - Hook Change Manager to Insert Overwrite
* [HIVE-16647] - Improve the validation output to make the output to stderr and stdout more consistent
* [HIVE-16651] - LlapProtocolClientProxy stack trace when using llap input format
* [HIVE-16652] - LlapInputFormat: Seeing "output error" WARN message
* [HIVE-16653] - Mergejoin should give itself a correct tag
* [HIVE-16672] - Parquet vectorization doesn't work for tables with partition info
* [HIVE-16684] - Bootstrap REPL DUMP shouldn't fail when table is dropped after fetching the table names.
* [HIVE-16686] - repl invocations of distcp needs additional handling
* [HIVE-16688] - Make sure Alter Table to set transaction=true acquires X lock
* [HIVE-16691] - Add test for more datatypes for LlapInputFormat
* [HIVE-16697] - Schema table validator should return a sorted list of missing tables
* [HIVE-16702] - Use LazyBinarySerDe for LLAP InputFormat
* [HIVE-16706] - Bootstrap REPL DUMP shouldn't fail when a partition is dropped/renamed when dump in progress.
* [HIVE-16714] - make Task Dependency on Repl Load more intuitive
* [HIVE-16715] - Clean up javadoc from errors in modules llap-client, metastore, spark-client
* [HIVE-16722] - Converting bucketed non-acid table to acid should perform validation
* [HIVE-16727] - REPL DUMP for insert event should't fail if the table is already dropped.
* [HIVE-16729] - Improve location validator to check for blank paths.
* [HIVE-16747] - Remove YETUS*.sh files after a YETUS release
* [HIVE-16748] - Integreate YETUS to Pre-Commit
* [HIVE-16750] - Support change management for rename table/partition.
* [HIVE-16764] - Support numeric as same as decimal
* [HIVE-16765] - ParquetFileReader should be closed to avoid resource leak
* [HIVE-16774] - Support position in ORDER BY when using SELECT *
* [HIVE-16775] - Fix HiveFilterAggregateTransposeRule when filter is always false
* [HIVE-16779] - CachedStore leak PersistenceManager resources
* [HIVE-16782] - Flaky Test: TestMiniLlapLocalCliDriver[subquery_scalar]
* [HIVE-16785] - Ensure replication actions are idempotent if any series of events are applied again.
* [HIVE-16797] - Enhance HiveFilterSetOpTransposeRule to remove union branches
* [HIVE-16813] - Incremental REPL LOAD should load the events in the same sequence as it is dumped.
* [HIVE-16827] - Merge stats task and column stats task into a single task
* [HIVE-16837] - MetadataOnly optimizer conflicts with count distinct rewrite
* [HIVE-16838] - Improve plans for subqueries with non-equi co-related predicates
* [HIVE-16848] - NPE during CachedStore refresh
* [HIVE-16892] - Move creation of _files from ReplCopyTask to analysis phase for boostrap replication
* [HIVE-16893] - move replication dump related work in semantic analysis phase to execution phase using a task
* [HIVE-16895] - Multi-threaded execution of bootstrap dump of partitions
* [HIVE-16896] - move replication load related work in semantic analysis phase to execution phase using a task
* [HIVE-16901] - Distcp optimization - One distcp per ReplCopyTask
* [HIVE-16912] - Improve table validator's performance against Oracle
* [HIVE-16926] - LlapTaskUmbilicalExternalClient should not start new umbilical server for every fragment request
* [HIVE-16974] - Change the sort key for the schema tool validator to be <ID>
* [HIVE-16981] - hive.optimize.bucketingsorting should compare the schema before removing RS
* [HIVE-16990] - REPL LOAD should update last repl ID only after successful copy of data files.
* [HIVE-16992] - LLAP: monitoring and better default lambda for LRFU policy
* [HIVE-16996] - Add HLL as an alternative to FM sketch to compute stats
* [HIVE-16997] - Extend object store to store and use bit vectors
* [HIVE-16998] - Add config to enable HoS DPP only for map-joins
* [HIVE-17005] - Ensure REPL DUMP and REPL LOAD are authorized properly
* [HIVE-17021] - Support replication of concatenate operation.
* [HIVE-17087] - Remove unnecessary HoS DPP trees during map-join conversion
* [HIVE-17091] - "Timed out getting readerEvents" error from external LLAP client
* [HIVE-17100] - Improve HS2 operation logs for REPL commands.
* [HIVE-17112] - Reduce logging in HiveSparkClientFactory and RemoteHiveSparkClient
* [HIVE-17132] - Add InterfaceAudience and InterfaceStability annotations for UDF APIs
* [HIVE-17137] - Fix javolution conflict
* [HIVE-17153] - Flaky test: TestMiniSparkOnYarnCliDriver[spark_dynamic_partition_pruning]
* [HIVE-17157] - Add InterfaceAudience and InterfaceStability annotations for ObjectInspector APIs
* [HIVE-17167] - Create metastore specific configuration tool
* [HIVE-17168] - Create separate module for stand alone metastore
* [HIVE-17170] - Move thrift generated code to stand alone metastore
* [HIVE-17178] - Spark Partition Pruning Sink Operator can't target multiple Works
* [HIVE-17183] - Disable rename operations during bootstrap dump
* [HIVE-17185] - TestHiveMetaStoreStatsMerge.testStatsMerge is failing
* [HIVE-17195] - Long chain of tasks created by REPL LOAD shouldn't cause stack corruption.
* [HIVE-17196] - CM: ReplCopyTask should retain the original file names even if copied from CM path.
* [HIVE-17205] - add functional support for unbucketed tables
* [HIVE-17212] - Dynamic add partition by insert shouldn't generate INSERT event.
* [HIVE-17214] - check/fix conversion of unbucketed non-acid to acid
* [HIVE-17215] - Streaming Ingest API writing unbucketed tables
* [HIVE-17216] - Additional qtests for HoS DPP
* [HIVE-17224] - Move JDO classes to standalone metastore
* [HIVE-17225] - HoS DPP pruning sink ops can target parallel work objects
* [HIVE-17241] - Change metastore classes to not use the shims
* [HIVE-17247] - HoS DPP: UDFs on the partition column side does not evaluate correctly
* [HIVE-17256] - add a notion of a guaranteed task to LLAP
* [HIVE-17289] - EXPORT and IMPORT shouldn't perform distcp with doAs privileged user.
* [HIVE-17292] - Change TestMiniSparkOnYarnCliDriver test configuration to use the configured cores
* [HIVE-17297] - allow AM to use LLAP guaranteed tasks
* [HIVE-17307] - Change the metastore to not use the metrics code in hive/common
* [HIVE-17316] - Use String.startsWith for the hidden configuration variables
* [HIVE-17318] - Make Hikari CP configurable using hive properties in hive-site.xml
* [HIVE-17319] - Make BoneCp configurable using hive properties in hive-site.xml
* [HIVE-17330] - refactor TezSessionPoolManager to separate its multiple functions
* [HIVE-17346] - TestMiniSparkOnYarnCliDriver[spark_dynamic_partition_pruning] is failing every time
* [HIVE-17347] - TestMiniSparkOnYarnCliDriver[spark_dynamic_partition_pruning_mapjoin_only] is failing every time
* [HIVE-17359] - Deal with TypeInfo dependencies in the metastore
* [HIVE-17371] - Move tokenstores to metastore module
* [HIVE-17375] - stddev_samp,var_samp standard compliance
* [HIVE-17380] - refactor LlapProtocolClientProxy to be usable with other protocols
* [HIVE-17381] - When we enable Parquet Writer Version V2, hive throws an exception: Unsupported encoding: DELTA_BYTE_ARRAY.
* [HIVE-17382] - Change startsWith relation introduced in HIVE-17316
* [HIVE-17387] - implement Tez AM registry in Hive
* [HIVE-17405] - HoS DPP ConstantPropagate should use ConstantPropagateOption.SHORTCUT
* [HIVE-17409] - refactor LLAP ZK registry to make the ZK-registry part reusable
* [HIVE-17414] - HoS DPP + Vectorization generates invalid explain plan due to CombineEquivalentWorkResolver
* [HIVE-17428] - REPL LOAD of ALTER_PARTITION event doesn't create import tasks if the partition doesn't exist during analyze phase.
* [HIVE-17455] - External LLAP client: connection to HS2 should be kept open until explicitly closed
* [HIVE-17456] - Set current database for external LLAP interface
* [HIVE-17473] - implement workload management pools
* [HIVE-17482] - External LLAP client: acquire locks for tables queried directly by LLAP
* [HIVE-17488] - Move first set of classes to standalone metastore
* [HIVE-17494] - Bootstrap REPL DUMP throws exception if a partitioned table is dropped while reading partitions.
* [HIVE-17495] - CachedStore: prewarm improvement (avoid multiple sql calls to read partition column stats), refactoring and caching some aggregate stats
* [HIVE-17506] - Fix standalone-metastore pom.xml to not depend on hive's main pom
* [HIVE-17508] - Implement global execution triggers based on counters
* [HIVE-17514] - Use SHA-256 for cookie signer to improve security
* [HIVE-17515] - Use SHA-256 for GenericUDFMaskHash to improve security
* [HIVE-17527] - Support replication for rename/move table across database
* [HIVE-17528] - Add more q-tests for Hive-on-Spark with Parquet vectorized reader
* [HIVE-17534] - Add a config to turn off parquet vectorization
* [HIVE-17537] - Move Warehouse class to standalone metastore
* [HIVE-17541] - Move testing related methods from MetaStoreUtils to some testing related utility
* [HIVE-17566] - Create schema required for workload management.
* [HIVE-17581] - Replace some calcite dependencies with native ones
* [HIVE-17607] - remove ColumnStatsDesc usage from columnstatsupdatetask
* [HIVE-17608] - REPL LOAD should overwrite the data files if exists instead of duplicating it
* [HIVE-17617] - Rollup of an empty resultset should contain the grouping of the empty grouping set
* [HIVE-17629] - CachedStore - wait for prewarm at use time, not init time
* [HIVE-17645] - MM tables patch conflicts with HIVE-17482 (Spark/Acid integration)
* [HIVE-17647] - DDLTask.generateAddMmTasks(Table tbl) and other random code should not start transactions
* [HIVE-17651] - TableScanOperator might miss vectorization on flag
* [HIVE-17652] - retire ANALYZE TABLE ... PARTIALSCAN
* [HIVE-17661] - DBTxnManager.acquireLocks() - MM tables should use shared lock for Insert
* [HIVE-17671] - TableScanDesc.isAcidTable is restricted to FullAcid tables
* [HIVE-17681] - Need to log bootstrap dump progress state property to HS2 logs.
* [HIVE-17692] - Block HCat on Acid tables
* [HIVE-17696] - Vectorized reader does not seem to be pushing down projection columns in certain code paths
* [HIVE-17698] - FileSinkDesk.getMergeInputDirName() uses stmtId=0
* [HIVE-17708] - Upgrade surefire to 2.20.1
* [HIVE-17728] - TestHCatClient should use hive.metastore.transactional.event.listeners as per recommendation.
* [HIVE-17733] - Move RawStore to standalone metastore
* [HIVE-17743] - Add InterfaceAudience and InterfaceStability annotations for Thrift generated APIs
* [HIVE-17748] - ReplCopyTask doesn't support multi-file CopyWork
* [HIVE-17750] - add a flag to automatically create most tables as MM
* [HIVE-17756] - Enable subquery related Qtests for Hive on Spark
* [HIVE-17757] - REPL LOAD need to use customised configurations to execute distcp/remote copy.
* [HIVE-17771] - Implement commands to manage resource plan
* [HIVE-17778] - Add support for custom counters in trigger expression
* [HIVE-17809] - Implement per pool trigger validation and move sessions across pools
* [HIVE-17812] - Move remaining classes that HiveMetaStore depends on
* [HIVE-17835] - HS2 Logs print unnecessary stack trace when HoS query is cancelled
* [HIVE-17837] - Explicitly check if the HoS Remote Driver has been lost in the RemoteSparkJobMonitor
* [HIVE-17841] - implement applying the resource plan
* [HIVE-17842] - Run checkstyle on ptest2 module with proper configuration
* [HIVE-17850] - can VectorizedOrcAcidRowReader be removed once HIVE-17458 is done?
* [HIVE-17856] - MM tables - IOW is not ACID compliant
* [HIVE-17858] - MM - some union cases are broken
* [HIVE-17874] - Parquet vectorization fails on tables with complex columns when there are no projected columns
* [HIVE-17884] - Implement create, alter and drop workload management triggers
* [HIVE-17887] - Incremental REPL LOAD with Drop partition event on timestamp type partition column fails.
* [HIVE-17888] - Display the reason for query cancellation
* [HIVE-17897] - "repl load" in bootstrap phase fails when partitions have whitespace
* [HIVE-17902] - add notions of default pool and start adding unmanaged mapping
* [HIVE-17904] - handle internal Tez AM restart in registry and WM
* [HIVE-17905] - propagate background LLAP cluster changes to WM
* [HIVE-17906] - use kill query mechanics to kill queries in WM
* [HIVE-17907] - enable and apply resource plan commands in HS2
* [HIVE-17913] - Cleanup unused methods in Driver
* [HIVE-17926] - Support triggers for non-pool sessions
* [HIVE-17929] - Use sessionId for HoS Remote Driver Client id
* [HIVE-17931] - Implement Parquet vectorization reader for Array type
* [HIVE-17933] - make antlr output directory to use a top-level sourceset
* [HIVE-17934] - Merging Statistics are promoted to COMPLETE (most of the time)
* [HIVE-17945] - Support column projection for index access when using Parquet Vectorization
* [HIVE-17950] - Implement resource plan fetching from metastore
* [HIVE-17954] - Implement pool, user, group and trigger to pool management API's.
* [HIVE-17961] - NPE during initialization of VectorizedParquetRecordReader when input split is null
* [HIVE-17967] - Move HiveMetaStore class
* [HIVE-17970] - MM LOAD DATA with OVERWRITE doesn't use base_n directory concept
* [HIVE-17972] - Implement Parquet vectorization reader for Map type
* [HIVE-17980] - Move HiveMetaStoreClient plus a few remaining classes.
* [HIVE-17981] - Create a set of builders for Thrift classes
* [HIVE-17982] - Move metastore specific itests
* [HIVE-17983] - Make the standalone metastore generate tarballs etc.
* [HIVE-17990] - Add Thrift and DB storage for Schema Registry objects
* [HIVE-17991] - Remove CommandNeedRetryException
* [HIVE-17995] - Run checkstyle on standalone-metastore module with proper configuration
* [HIVE-17996] - Fix ASF headers
* [HIVE-17997] - Add rat plugin and configuration to standalone metastore pom
* [HIVE-18002] - add group support for pool mappings
* [HIVE-18003] - add explicit jdbc connection string args for mappings
* [HIVE-18004] - investigate deriving app name from JDBC connection for pool mapping
* [HIVE-18005] - Improve size estimation for array() to be not 0
* [HIVE-18025] - Push resource plan changes to tez/unmanaged sessions
* [HIVE-18028] - fix WM based on cluster smoke test; add logging
* [HIVE-18029] - beeline - support proper usernames based on the URL arg
* [HIVE-18031] - Support replication for Alter Database operation.
* [HIVE-18034] - Improving logging with HoS executors spend lots of time in GC
* [HIVE-18036] - Stats: Remove usage of clone() methods
* [HIVE-18053] - Support different table types for MVs
* [HIVE-18056] - CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
* [HIVE-18057] - remove PostExecute / PreExecute hook support
* [HIVE-18063] - Make CommandProcessorResponse an exception instead of a return class
* [HIVE-18071] - add HS2 jmx information about pools and current resource plan
* [HIVE-18072] - fix various WM bugs based on cluster testing - part 2
* [HIVE-18073] - AM may assert when its guaranteed task count is reduced
* [HIVE-18075] - verify commands on a cluster
* [HIVE-18076] - killquery doesn't actually work for non-trigger WM kills
* [HIVE-18078] - WM getSession needs some retry logic
* [HIVE-18084] - Upgrade checkstyle version to support lambdas
* [HIVE-18085] - Run checkstyle on storage-api module with proper configuration
* [HIVE-18088] - Add WM event traces at query level for debugging
* [HIVE-18092] - Fix exception on tables handled by HBaseHandler if columnsstats are auto-gathered
* [HIVE-18093] - Improve logging when HoS application is killed
* [HIVE-18095] - add a unmanaged flag to triggers (applies to container based sessions)
* [HIVE-18096] - add a user-friendly show plan command
* [HIVE-18125] - Support arbitrary file names in input to Load Data
* [HIVE-18133] - Parametrize TestTxnNoBuckets wrt Vectorization
* [HIVE-18134] - some alter resource plan fixes
* [HIVE-18138] - Fix columnstats problem in case schema evolution
* [HIVE-18141] - Fix StatsUtils.combineRange to combine intervals
* [HIVE-18149] - Stats: rownum estimation from datasize underestimates in most cases
* [HIVE-18153] - refactor reopen and file management in TezTask
* [HIVE-18161] - Remove hive.stats.atomic
* [HIVE-18163] - Stats: create materialized view should also collect stats
* [HIVE-18170] - User mapping not initialized correctly on start
* [HIVE-18179] - Implement validate resource plan (part 1)
* [HIVE-18187] - Add jamon generated-sources as source folder
* [HIVE-18190] - Consider looking at ORC file schema rather than using _metadata_acid file
* [HIVE-18192] - Introduce WriteID per table rather than using global transaction ID
* [HIVE-18193] - Migrate existing ACID tables to use write id per table rather than global transaction id
* [HIVE-18202] - Automatically migrate hbase.table.name to hbase.mapreduce.hfileoutputformat.table.name for hbase-based table
* [HIVE-18203] - change the way WM is enabled and allow dropping the last resource plan
* [HIVE-18209] - Fix API call in VectorizedListColumnReader to get value from BytesColumnVector
* [HIVE-18211] - Support to read multiple level definition for Map type in Parquet file
* [HIVE-18212] - Make sure Yetus check always has a full log
* [HIVE-18214] - Flaky test: TestSparkClient
* [HIVE-18222] - Update checkstyle rules to be less peeky
* [HIVE-18224] - Introduce interface above driver
* [HIVE-18229] - add the unmanaged mapping command
* [HIVE-18230] - create plan like plan, and replace plan commands for easy modification
* [HIVE-18235] - Columnstats gather on mm tables: re-enable disabled test
* [HIVE-18237] - missing results for insert_only table after DP insert
* [HIVE-18238] - Driver execution may not have configuration changing sideeffects
* [HIVE-18245] - clean up acid_vectorization_original.q
* [HIVE-18257] - implement scheduling policy configuration instead of hardcoding fair scheduling
* [HIVE-18273] - add LLAP-level counters for WM
* [HIVE-18274] - add AM level metrics for WM
* [HIVE-18275] - add HS2-level WM metrics
* [HIVE-18286] - java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.vector.DoubleColumnVector cannot be cast to org.apache.hadoop.hive.ql.exec.vector.LongColumnVector
* [HIVE-18288] - merge/concat not supported on Acid table
* [HIVE-18294] - add switch to make acid table the default
* [HIVE-18315] - update tests use non-acid tables
* [HIVE-18317] - Improve error messages in TransactionalValidationListerner
* [HIVE-18323] - Vectorization: add the support of timestamp in VectorizedPrimitiveColumnReader for parquet
* [HIVE-18366] - Update HBaseSerDe to use hbase.mapreduce.hfileoutputformat.table.name instead of hbase.table.name as the table name property
* [HIVE-18368] - Improve Spark Debug RDD Graph
* [HIVE-18372] - Create testing infra to test different HMS instances
* [HIVE-18389] - Print out Spark Web UI URL to the console log
* [HIVE-18411] - Fix ArrayIndexOutOfBoundsException for VectorizedListColumnReader
* [HIVE-18418] - clean up plugin between DAGs
* [HIVE-18437] - use plan parallelism for the default pool if both are present
* [HIVE-18438] - WM RP: it's impossible to unset things
* [HIVE-18443] - Ensure git gc finished in ptest prep phase before copying repo
* [HIVE-18457] - improve show plan output (triggers, mappings)
* [HIVE-18458] - Workload manager initializes even when interactive queue is not set
* [HIVE-18468] - Create tests to cover alterPartition and renamePartition methods
* [HIVE-18478] - Data files deleted from temp table should not be recycled to CM path
* [HIVE-18479] - Create tests to cover dropPartition methods
* [HIVE-18480] - Create tests for function related methods
* [HIVE-18481] - Create tests for table related methods (get, list, exists)
* [HIVE-18483] - Create tests to cover getPartition(s) methods
* [HIVE-18484] - Create tests to cover listPartition(s) methods
* [HIVE-18486] - Create tests to cover add partition methods
* [HIVE-18489] - Automatically migrate s3n URIs to s3a URIs
* [HIVE-18495] - JUnit rule to enable Driver level testing
* [HIVE-18496] - Create tests to cover add/alter/drop index methods
* [HIVE-18498] - Create tests to cover get and list index methods
* [HIVE-18509] - Create tests for table manipulation related methods (create, alter, drop)
* [HIVE-18511] - Fix generated checkstyle errors
* [HIVE-18536] - IOW + DP is broken for insert-only ACID
* [HIVE-18541] - Secure HS2 web UI with PAM
* [HIVE-18542] - Create tests to cover getTableMeta method
* [HIVE-18544] - Create tests to cover appendPartition methods
* [HIVE-18550] - Keep the hbase table name property as hbase.table.name
* [HIVE-18553] - Support schema evolution in Parquet Vectorization reader
* [HIVE-18566] - Create tests to cover adding partitions from PartitionSpec
* [HIVE-18580] - Create tests to cover exchange partitions
* [HIVE-18596] - Synchronize value of hive.spark.client.connect.timeout across unit tests
* [HIVE-18609] - Results cache invalidation based on ACID table updates
* [HIVE-18633] - Service discovery for Active/Passive HA mode
* [HIVE-18635] - Generalize hook dispatch logics in Driver
* [HIVE-18651] - Expose additional Spark metrics
* [HIVE-18663] - Logged Spark Job Id contains a UUID instead of the actual id
* [HIVE-18672] - Printed state in RemoteSparkJobMonitor is ambiguous
* [HIVE-18673] - ErrorMsg.SPARK_JOB_MONITOR_TIMEOUT isn't formatted correctly
* [HIVE-18677] - SparkClientImpl usage of SessionState.LogHelper doesn't respect isSilent value
* [HIVE-18679] - create/replicate open transaction event
* [HIVE-18703] - Make Operator comparision to be based on some primitive
* [HIVE-18715] - Remove index support from metastore
* [HIVE-18720] - Replicate Commit Txn operation (without writes)
* [HIVE-18745] - Fix MetaStore creation in tests, so multiple MetaStores can be started on the same machine
* [HIVE-18747] - Cleaner for TXN_TO_WRITE_ID table entries using MIN_HISTORY_LEVEL.
* [HIVE-18749] - Need to replace transactionId with writeId in RecordIdentifier and other relevant contexts.
* [HIVE-18750] - Exchange partition should be disabled on ACID/Insert-only tables with per table write ID.
* [HIVE-18751] - ACID table scan through get_splits UDF doesn't receive ValidWriteIdList configuration.
* [HIVE-18753] - Correct methods and variables names which uses writeId instead of transactionId.
* [HIVE-18755] - Modifications to the metastore for catalogs
* [HIVE-18765] - SparkClientImpl swallows exception messages from the RemoteDriver
* [HIVE-18771] - Refactor tests, so only 1 MetaStore instance will be started per test class and test configuration
* [HIVE-18781] - Create/Replicate Open, Commit (without writes) and Abort Txn events
* [HIVE-18805] - Add ConstantPropagate before stats annotation
* [HIVE-18824] - ValidWriteIdList config should be defined on tables which has to collect stats after insert
* [HIVE-18830] - RemoteSparkJobMonitor failures are logged twice
* [HIVE-18832] - Support change management for trashing data files from ACID tables.
* [HIVE-18840] - CachedStore: Prioritize loading of recently accessed tables during prewarm
* [HIVE-18846] - Query results cache: Allow queries to refer to the pending results of a query that has not finished yet
* [HIVE-18855] - Fix unit test TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]
* [HIVE-18861] - druid-hdfs-storage is pulling in hadoop-aws-2.7.x and aws SDK, creating classpath problems on hadoop 3.x
* [HIVE-18864] - ValidWriteIdList snapshot seems incorrect if obtained after allocating writeId by current transaction.
* [HIVE-18899] - Separate FetchWork required for each query that uses the results cache
* [HIVE-18909] - Metrics for results cache
* [HIVE-18926] - Imporve operator-tree matching
* [HIVE-18946] - Fix columnstats merge NPE
* [HIVE-18961] - Error in results cache when query has identifiers with spaces
* [HIVE-18982] - Provide a CLI option to manually trigger failover
* [HIVE-18988] - Support bootstrap replication of ACID tables
* [HIVE-18994] - Handle client connections on failover
* [HIVE-19009] - Retain and use runtime statistics during hs2 lifetime
* [HIVE-19031] - Mark duplicate configs in HiveConf as deprecated
* [HIVE-19083] - Make partition clause optional for INSERT
* [HIVE-19089] - Create/Replicate Allocate write-id event
* [HIVE-19112] - Support Analyze table for partitioned tables without partition spec
* [HIVE-19126] - CachedStore: Use memory estimation to limit cache size during prewarm
* [HIVE-19127] - Concurrency fixes in QueryResultsCache
* [HIVE-19128] - Update golden files for spark perf tests
* [HIVE-19129] - Support DEFAULT keyword with MERGE
* [HIVE-19135] - Need tool to allow admins to create catalogs and move existing dbs to catalog during upgrade
* [HIVE-19138] - Results cache: allow queries waiting on pending cache entries to check cache again if pending query fails
* [HIVE-19141] - TestNegativeCliDriver insert_into_notnull_constraint, insert_into_acid_notnull failing
* [HIVE-19144] - TestSparkCliDriver:subquery_scalar - golden file needs to be udpated
* [HIVE-19145] - Stabilize statsoptimizer.q test
* [HIVE-19146] - Delete dangling q.out
* [HIVE-19147] - Fix PerfCliDrivers: Tpcds30T missed CAT_NAME change
* [HIVE-19153] - Update golden files for few tests
* [HIVE-19154] - Poll notification events to invalidate the results cache
* [HIVE-19156] - TestMiniLlapLocalCliDriver.vectorized_dynamic_semijoin_reduction.q is broken
* [HIVE-19159] - TestMTQueries.testMTQueries1 failure
* [HIVE-19164] - TestMetastoreVersion failures
* [HIVE-19171] - Persist runtime statistics in metastore
* [HIVE-19175] - TestMiniLlapLocalCliDriver.testCliDriver update_access_time_non_current_db failing
* [HIVE-19178] - TestMiniTezCliDriver.testCliDriver[explainanalyze_5] failure
* [HIVE-19193] - TestActivePassiveHA fails
* [HIVE-19194] - TestDruidStorageHandler fails
* [HIVE-19195] - Fix flaky tests and cleanup testconfiguration to run llap specific tests in llap only.
* [HIVE-19196] - TestTriggersMoveWorkloadManager is flaky
* [HIVE-19197] - TestReplicationScenarios is flaky
* [HIVE-19206] - Automatic memory management for open streaming writers
* [HIVE-19209] - Streaming ingest record writers should accept input stream
* [HIVE-19210] - Create separate module for streaming ingest
* [HIVE-19211] - New streaming ingest API and support for dynamic partitioning
* [HIVE-19214] - High throughput ingest ORC format
* [HIVE-19222] - TestNegativeCliDriver tests are failing due to "java.lang.OutOfMemoryError: GC overhead limit exceeded"
* [HIVE-19232] - results_cache_invalidation2 is failing
* [HIVE-19274] - Add an OpTreeSignature persistence checker hook
* [HIVE-19319] - RuntimeStats fixes
* [HIVE-19322] - broken test: TestNegativeMinimrCliDriver#testCliDriver[minimr_broken_pipe]
* [HIVE-19335] - Disable runtime filtering (semijoin reduction opt with bloomfilter) for external tables
* [HIVE-19346] - TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_5] failling
* [HIVE-19347] - TestTriggersWorkloadManager tests are failing consistently
* [HIVE-19348] - org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp are failing
* [HIVE-19371] - Add table ownerType to HMS thrift API
* [HIVE-19372] - Add table ownerType to JDO/SQL and ObjectStore
* [HIVE-19374] - Parse and process ALTER TABLE SET OWNER command syntax
* [HIVE-19400] - Adjust Hive 1.0 to 2.0 conversion utility to the upgrade
* [HIVE-19471] - bucket_map_join_tez1 and bucket_map_join_tez2 are failing
* [HIVE-19472] - HiveStreamingConnection swallows exception on partition creation
* [HIVE-19494] - Accept shade prefix during reflective instantiation of output format
** Bug
* [HIVE-4577] - hive CLI can't handle hadoop dfs command with space and quotes.
* [HIVE-6348] - Order by/Sort by in subquery
* [HIVE-6590] - Hive does not work properly with boolean partition columns (wrong results and inserts to incorrect HDFS path)
* [HIVE-6990] - Direct SQL fails when the explicit schema setting is different from the default one
* [HIVE-8937] - fix description of hive.security.authorization.sqlstd.confwhitelist.* params
* [HIVE-9815] - Metastore column"SERDE_PARAMS"."PARAM_VALUE" limited to 4000 bytes
* [HIVE-10616] - TypeInfoUtils doesn't handle DECIMAL with just precision specified
* [HIVE-10865] - Beeline needs to support DELIMITER command
* [HIVE-11064] - ALTER TABLE CASCADE ERROR unbalanced calls to openTransaction/commitTransaction
* [HIVE-11266] - count(*) wrong result based on table statistics for external tables
* [HIVE-11297] - Combine op trees for partition info generating tasks
* [HIVE-11609] - Capability to add a filter to hbase scan via composite key doesn't work
* [HIVE-12408] - SQLStdAuthorizer should not require external table creator to be owner of directory, in addition to rw permissions
* [HIVE-12425] - OrcRecordUpdater.close(true) leaves the file open
* [HIVE-12631] - LLAP IO: support ORC ACID tables
* [HIVE-12719] - As a hive user, I am facing issues using permanent UDAF's.
* [HIVE-12734] - Remove redundancy in HiveConfs serialized to UDFContext
* [HIVE-13000] - Hive returns useless parsing error
* [HIVE-13652] - Import table change order of dynamic partitions
* [HIVE-14032] - INSERT OVERWRITE command failed with case sensitive partition key names
* [HIVE-14052] - Cleanup structures when external clients use LLAP
* [HIVE-14077] - add implicit decimal arithmetic q test, fix issues if found
* [HIVE-14455] - upgrade httpclient, httpcore to match updated hadoop dependency
* [HIVE-14560] - Support exchange partition between s3 and hdfs tables
* [HIVE-14564] - Column Pruning generates out of order columns in SelectOperator which cause ArrayIndexOutOfBoundsException.
* [HIVE-14678] - Hive-on-MR deprecation warning is not diplayed when engine is set to capital letter 'MR'
* [HIVE-14731] - Use Tez cartesian product edge in Hive (unpartitioned case only)
* [HIVE-14792] - AvroSerde reads the remote schema-file at least once per mapper, per table reference.
* [HIVE-14813] - Make TransactionBatchImpl.toString() include state of each txn: commit/abort
* [HIVE-14988] - Support INSERT OVERWRITE into a partition on transactional tables
* [HIVE-15077] - Acid LockManager is unfair
* [HIVE-15104] - Hive on Spark generate more shuffle data than hive on mr
* [HIVE-15144] - JSON.org license is now CatX
* [HIVE-15160] - Can't order by an unselected column
* [HIVE-15176] - Small typo in hiveserver2 webui
* [HIVE-15249] - HIve 2.1.0 is throwing InvalidObjectException(message:Invalid column type name is too long
* [HIVE-15267] - Make query length calculation logic more accurate in TxnUtils.needNewQuery()
* [HIVE-15343] - Spelling errors in logging and exceptions for beeline, common, hbase-handler, hcatalog, llap-server, orc, serde and shims
* [HIVE-15344] - Spelling errors in logging and exceptions for metastore and service directories
* [HIVE-15442] - Driver.java has a redundancy code
* [HIVE-15483] - Database and table name is case sensitive when used in show grant
* [HIVE-15504] - ArrayIndexOutOfBoundsException in GenericUDFTrunc::initialize
* [HIVE-15515] - Remove the docs directory
* [HIVE-15552] - Unable to coalesce DATE and TIMESTAMP types
* [HIVE-15630] - add operation handle before operation.run instead of after operation.run
* [HIVE-15632] - Hive/Druid integration: Incorrect result - Limit on timestamp disappears
* [HIVE-15635] - Hive/Druid integration: timeseries query shows all days, even if no data
* [HIVE-15636] - Hive/Druid integration: wrong semantics of topN query limit with granularity
* [HIVE-15637] - Hive/Druid integration: wrong semantics of groupBy query limit with granularity
* [HIVE-15639] - Hive/Druid integration: wrong semantics for ordering within groupBy queries
* [HIVE-15680] - Incorrect results when hive.optimize.index.filter=true and same ORC table is referenced twice in query
* [HIVE-15724] - getPrimaryKeys and getForeignKeys in metastore does not normalize db and table name
* [HIVE-15739] - Incorrect exception message in PartExprEvalUtils
* [HIVE-15761] - ObjectStore.getNextNotification could return an empty NotificationEventResponse causing TProtocolException
* [HIVE-15767] - Hive On Spark is not working on secure clusters from Oozie
* [HIVE-15829] - LLAP text cache: disable memory tracking on the writer
* [HIVE-15883] - HBase mapped table in Hive insert fail for decimal
* [HIVE-15995] - Syncing metastore table with serde schema
* [HIVE-16007] - When the query does not complie the LogRunnable never stops
* [HIVE-16025] - Where IN clause throws exception
* [HIVE-16026] - Generated query will timeout and/or kill the druid cluster.
* [HIVE-16027] - <timestamp> BETWEEN <string> AND <string> must cast to TIMESTMAP
* [HIVE-16044] - LLAP: Shuffle Handler keep-alive connections are closed from the server side
* [HIVE-16053] - Remove newRatio from llap JAVA_OPTS_BASE
* [HIVE-16057] - SchemaTool ignores --passWord argument if hadoop.security.credential.provider.path is configured
* [HIVE-16061] - When hive.async.log.enabled is set to true, some output is not printed to the beeline console
* [HIVE-16077] - UPDATE/DELETE fails with numBuckets > numReducers
* [HIVE-16113] - PartitionPruner::removeNonPartCols needs to handle AND/OR cases
* [HIVE-16117] - SortProjectTransposeRule should check for monotonicity preserving CAST
* [HIVE-16125] - Split work between reducers.
* [HIVE-16130] - Remove jackson classes from hive-jdbc standalone jar
* [HIVE-16147] - Rename a partitioned table should not drop its partition columns stats
* [HIVE-16174] - Update MetricsConstant.WAITING_COMPILE_OPS metric when we aquire lock failed in Driver
* [HIVE-16177] - non Acid to acid conversion doesn't handle _copy_N files
* [HIVE-16188] - beeline should block the connection if given invalid database name.
* [HIVE-16193] - Hive show compactions not reflecting the status of the application
* [HIVE-16213] - ObjectStore can leak Queries when rollbackTransaction throws an exception
* [HIVE-16219] - metastore notification_log contains serialized message with non functional fields
* [HIVE-16222] - add a setting to disable row.serde for specific formats; enable for others
* [HIVE-16225] - Memory leak in webhcat service (FileSystem CACHE entries)
* [HIVE-16233] - llap: Query failed with AllocatorOutOfMemoryException
* [HIVE-16254] - metadata for values temporary tables for INSERTs are getting replicated during bootstrap
* [HIVE-16275] - Vectorization: Add ReduceSink support for TopN (in specialized native classes)
* [HIVE-16282] - Semijoin: Disable slow-start for the bloom filter aggregate task
* [HIVE-16287] - Alter table partition rename with location - moves partition back to hive warehouse
* [HIVE-16290] - Stats: StatsRulesProcFactory::evaluateComparator estimates are wrong when minValue == filterValue
* [HIVE-16291] - Hive fails when unions a parquet table with itself
* [HIVE-16296] - use LLAP executor count to configure reducer auto-parallelism
* [HIVE-16298] - Add config to specify multi-column joins have correlated columns
* [HIVE-16299] - MSCK REPAIR TABLE should enforce partition key order when adding unknown partitions
* [HIVE-16302] - Add junit dependency to hive-shims-common to compile with Hadoop 2.8+
* [HIVE-16305] - Additional Datanucleus ClassLoaderResolverImpl leaks causing HS2 OOM
* [HIVE-16307] - add IO memory usage report to LLAP UI
* [HIVE-16308] - PreExecutePrinter and PostExecutePrinter should log to INFO level instead of ERROR
* [HIVE-16309] - Hive Test Commands failure should be printed in hive.log in addition to stderr
* [HIVE-16315] - Describe table doesn't show num of partitions
* [HIVE-16316] - Prepare master branch for 3.0.0 development.
* [HIVE-16317] - CASE .. NULL in JOIN condition can trigger SemanticException
* [HIVE-16318] - LLAP cache: address some issues in 2.2/2.3
* [HIVE-16319] - LLAP: Better handling of an empty wait queue, should try scheduling checks
* [HIVE-16321] - Possible deadlock in metastore with Acid enabled
* [HIVE-16323] - HS2 JDOPersistenceManagerFactory.pmCache leaks after HIVE-14204
* [HIVE-16324] - Truncate table should not work when EXTERNAL property of table is true
* [HIVE-16325] - Tez session refresh based on a time interval fails
* [HIVE-16328] - HoS: more aggressive mapjoin optimization when hive.spark.use.ts.stats.for.mapjoin is true
* [HIVE-16329] - TopN: use local executor info for LLAP memory checks
* [HIVE-16333] - remove the redundant symbol "\" to appear red in sublime text 3
* [HIVE-16335] - Beeline user HS2 connection file should use /etc/hive/conf instead of /etc/conf/hive
* [HIVE-16336] - Rename hive.spark.use.file.size.for.mapjoin to hive.spark.use.ts.stats.for.mapjoin
* [HIVE-16341] - Tez Task Execution Summary has incorrect input record counts on some operators
* [HIVE-16347] - HiveMetastoreChecker should skip listing partitions which are not valid when hive.msck.path.validation is set to skip or ignore
* [HIVE-16353] - Jetty 9 upgrade breaks hive master LLAP
* [HIVE-16357] - Failed folder creation when creating a new table is reported incorrectly
* [HIVE-16363] - QueryLifeTimeHooks should catch parse exceptions
* [HIVE-16368] - Unexpected java.lang.ArrayIndexOutOfBoundsException from query with LaterView Operation for hive on MR.
* [HIVE-16369] - Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only)
* [HIVE-16380] - removing global test dependency of jsonassert
* [HIVE-16384] - Remove jdk7 build from travis
* [HIVE-16385] - StatsNoJobTask could exit early before all partitions have been processed
* [HIVE-16388] - LLAP: Log rotation for daemon, history and gc files
* [HIVE-16389] - Allow HookContext to access SQLOperationDisplay
* [HIVE-16390] - LLAP IO should take job config into account; also LLAP config should load defaults
* [HIVE-16393] - Fix visibility of CodahaleReporter interface
* [HIVE-16394] - HoS does not support queue name change in middle of session
* [HIVE-16396] - Sync storage-api version in pom.xml
* [HIVE-16399] - create an index for tc_txnid in TXN_COMPONENTS
* [HIVE-16402] - Upgrade to Hadoop 2.8.0
* [HIVE-16403] - LLAP UI shows the wrong number of executors
* [HIVE-16404] - Renaming of public classes in Calcite 12 breeaking druid integration
* [HIVE-16406] - Remove unwanted interning when creating PartitionDesc
* [HIVE-16409] - TestEventHandlerFactory has lacked the ASF header
* [HIVE-16413] - Create table as select does not check ownership of the location
* [HIVE-16421] - Runtime filtering breaks user-level explain
* [HIVE-16422] - Should kill running Spark Jobs when a query is cancelled.
* [HIVE-16425] - Vectorization: unload old hashtables before reloadHashTable
* [HIVE-16427] - Fix multi-insert query and write qtests
* [HIVE-16433] - Not nullify variable "rj" to avoid NPE due to race condition in ExecDriver.
* [HIVE-16436] - Response times in "Task Execution Summary" at the end of the job is not correct
* [HIVE-16448] - Vectorization: Vectorized order_null.q fails with deserialize EOF exception below TEZ ReduceRecordSource.processVectorGroup
* [HIVE-16450] - Some metastore operations are not retried even with desired underlining exceptions
* [HIVE-16451] - Race condition between HiveStatement.getQueryLog and HiveStatement.runAsyncOnServer
* [HIVE-16459] - Forward channelInactive to RpcDispatcher
* [HIVE-16461] - DagUtils checks local resource size on the remote fs
* [HIVE-16462] - Vectorization: Enabling hybrid grace disables specialization of all reduce side joins
* [HIVE-16465] - NullPointer Exception when enable vectorization for Parquet file format
* [HIVE-16468] - BeeLineDriver should be able to run tests against an externally created cluster
* [HIVE-16471] - Add metrics for "waiting compilation time"
* [HIVE-16473] - Hive-on-Tez may fail to write to an HBase table
* [HIVE-16482] - Druid Ser/Des need to use dimension output name
* [HIVE-16483] - HoS should populate split related configurations to HiveConf
* [HIVE-16485] - Enable outputName for RS operator in explain formatted
* [HIVE-16487] - Serious Zookeeper exception is logged when a race condition happens
* [HIVE-16491] - CBO cant handle join involving complex types in on condition
* [HIVE-16494] - udaf percentile_approx() may fail on CBO
* [HIVE-16497] - FileUtils. isActionPermittedForFileHierarchy, isOwnerOfFileHierarchy file system operations should be impersonated
* [HIVE-16507] - Hive Explain User-Level may print out "Vertex dependency in root stage" twice
* [HIVE-16510] - Vectorization: Add vectorized PTF tests in preparation for HIVE-16369
* [HIVE-16511] - CBO looses inner casts on constants of complex type
* [HIVE-16513] - width_bucket issues
* [HIVE-16518] - Insert override for druid does not replace all existing segments
* [HIVE-16519] - Fix exception thrown by checkOutputSpecs
* [HIVE-16523] - VectorHashKeyWrapper hash code for strings is not so good
* [HIVE-16524] - Remove the redundant item type in hiveserver2.jsp and QueryProfileTmpl.jamon
* [HIVE-16533] - Vectorization: Avoid evaluating empty groupby keys
* [HIVE-16534] - Add capability to tell aborted transactions apart from open transactions in ValidTxnList
* [HIVE-16538] - TestExecDriver fails if run after TestOperators#testScriptOperator
* [HIVE-16539] - Add PTF tests for blobstores
* [HIVE-16545] - LLAP: bug in arena size determination logic
* [HIVE-16546] - LLAP: Fail map join tasks if hash table memory exceeds threshold
* [HIVE-16547] - LLAP: may not unlock buffers in some cases
* [HIVE-16553] - Change default value for hive.tez.bigtable.minsize.semijoin.reduction
* [HIVE-16554] - ACID: Make HouseKeeperService threads daemon
* [HIVE-16557] - Vectorization: Specialize ReduceSink empty key case
* [HIVE-16559] - Parquet schema evolution for partitioned tables may break if table and partition serdes differ
* [HIVE-16562] - Issues with nullif / fetch task
* [HIVE-16563] - Alter table partition set location should use fully qualified path for non-default FS
* [HIVE-16572] - Rename a partition should not drop its column stats
* [HIVE-16573] - In-place update for HoS can't be disabled
* [HIVE-16576] - Fix encoding of intervals when fetching select query candidates from druid
* [HIVE-16577] - Syntax error in the metastore init scripts for mssql
* [HIVE-16578] - Semijoin Hints should use column name, if provided for partition key check
* [HIVE-16581] - a bug in HIVE-16523
* [HIVE-16584] - Warning messages should use LogHelper.printInfo instead of printing to the infoStream directly
* [HIVE-16588] - Resource leak by druid http client
* [HIVE-16589] - Vectorization: Support Complex Types and GroupBy modes PARTIAL2, FINAL, and COMPLETE for AVG, VARIANCE
* [HIVE-16590] - Make initializing dag names in SparkWork thread safe for parallel compilation (HIVE-13512)
* [HIVE-16592] - Vectorization: Long hashCodes should bit-mix into lower bits
* [HIVE-16593] - SparkClientFactory.stop may prevent JVM from exiting
* [HIVE-16598] - LlapServiceDriver - create directories and warn of errors
* [HIVE-16599] - NPE in runtime filtering cost when handling SMB Joins
* [HIVE-16603] - Enforce foreign keys to refer to primary keys or unique keys
* [HIVE-16607] - ColumnStatsAutoGatherContext regenerates HiveConf.HIVEQUERYID
* [HIVE-16609] - col='__HIVE_DEFAULT_PARTITION__' condition in select statement may produce wrong result
* [HIVE-16610] - Semijoin Hint : Should be able to handle more than one hint per alias
* [HIVE-16613] - SaslClientHandler.sendHello is eating exceptions
* [HIVE-16625] - Extra '\0' characters in the output, when SeparatedValuesOutputFormat is used and the quoting is disabled
* [HIVE-16633] - username for ATS data shall always be the uid who submit the job
* [HIVE-16634] - LLAP Use a pool of connections to a single AM from a daemon
* [HIVE-16640] - The ASF Headers have some errors in some class
* [HIVE-16645] - Commands.java has missed the catch statement and has some code format errors
* [HIVE-16646] - Alias in transform ... as clause shouldn't be case sensitive
* [HIVE-16654] - Optimize a combination of avg(), sum(), count(distinct) etc
* [HIVE-16658] - TestTimestampTZ.java has missed the ASF header
* [HIVE-16659] - Query plan should reflect hive.spark.use.groupby.shuffle
* [HIVE-16660] - Not able to add partition for views in hive when sentry is enabled
* [HIVE-16665] - Race condition in Utilities.GetInputPathsCallable --> createDummyFileForEmptyPartition
* [HIVE-16667] - PostgreSQL metastore handling of CLOB types for COLUMNS_V2.TYPE_NAME and other field is incorrect
* [HIVE-16671] - LLAP IO: BufferUnderflowException may happen in very rare(?) cases due to ORC end-of-CB estimation
* [HIVE-16675] - Fix ConcurrentModificationException in SparkClientImpl#startDriver
* [HIVE-16677] - CTAS with no data fails in Druid
* [HIVE-16678] - Truncate on temporary table fails with "table not found" error.
* [HIVE-16679] - Missing ASF header on properties file in ptest2 project
* [HIVE-16689] - Correlated scalar subquery with comparison to constant in predicate fails
* [HIVE-16692] - LLAP: Keep alive connection in shuffle handler should not be closed until entire data is flushed out
* [HIVE-16693] - beeline "source" command freezes if you have a comment in it?
* [HIVE-16696] - Fix JoinCondDesc explain string
* [HIVE-16698] - HoS should avoid mapjoin optimization in case of union and using table stats
* [HIVE-16703] - Hive may add the same file to the session and vertex in Tez
* [HIVE-16708] - Exception while renewing a Delegation Token
* [HIVE-16721] - Inconsistent behavior in dealing with Timestamp stats
* [HIVE-16724] - increase session timeout for LLAP ZK token manager
* [HIVE-16730] - Vectorization: Schema Evolution for Text Vectorization / Complex Types
* [HIVE-16731] - Vectorization: Make "CASE WHEN (day_name='Sunday') THEN column1 ELSE null end" that involves a column name or expression THEN or ELSE vectorize
* [HIVE-16732] - Transactional tables should block LOAD DATA
* [HIVE-16737] - LLAP: Shuffle handler TCP listen queue overflows
* [HIVE-16738] - Notification ID generation in DBNotification might not be unique across HS2 instances.
* [HIVE-16742] - cap the number of reducers for LLAP at the configured value
* [HIVE-16743] - BitSet set() is incorrectly used in TxnUtils.createValidCompactTxnList()
* [HIVE-16744] - LLAP index update may be broken after ORC switch
* [HIVE-16745] - Syntax error in 041-HIVE-16556.mysql.sql script
* [HIVE-16746] - Reduce number of index lookups for same table in IndexWhereTaskDispatcher
* [HIVE-16751] - Support different types for grouping columns in GroupBy Druid queries
* [HIVE-16755] - LLAP IO: incorrect assert may trigger in tests
* [HIVE-16756] - Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
* [HIVE-16757] - Use of deprecated getRows() instead of new estimateRowCount(RelMetadataQuery..) has serious performance impact
* [HIVE-16761] - LLAP IO: SMB joins fail elevator
* [HIVE-16769] - Possible hive service startup due to the existing file /tmp/stderr
* [HIVE-16776] - Strange cast behavior for table backed by druid
* [HIVE-16777] - LLAP: Use separate tokens and UGI instances when an external client is used