| HDFS-15245 |
Improve JournalNode web UI |
Major |
journal-node, ui |
Jianfei Jiang |
Jianfei Jiang |
| HADOOP-16952 |
Add .diff to gitignore |
Minor |
. |
Ayush Saxena |
Ayush Saxena |
| HADOOP-16954 |
Add -S option in “Count” command to show only Snapshot Counts |
Major |
. |
Hemanth Boyina |
Hemanth Boyina |
| HDFS-15247 |
RBF: Provide Non DFS Used per DataNode in DataNode UI |
Major |
. |
Ayush Saxena |
Lisheng Sun |
| MAPREDUCE-7199 |
HsJobsBlock reuse JobACLsManager for checkAccess |
Minor |
. |
Bibin Chundatt |
Bilwa S T |
| HDFS-15295 |
AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. |
Minor |
. |
Jinglun |
Jinglun |
| HADOOP-16054 |
Update Dockerfile to use Bionic |
Major |
build, test |
Akira Ajisaka |
Akira Ajisaka |
| YARN-10237 |
Add isAbsoluteResource config for queue in scheduler response |
Minor |
scheduler |
Prabhu Joseph |
Prabhu Joseph |
| HADOOP-16886 |
Add hadoop.http.idle_timeout.ms to core-default.xml |
Major |
. |
Wei-Chiu Chuang |
Lisheng Sun |
| HDFS-14283 |
DFSInputStream to prefer cached replica |
Major |
. |
Wei-Chiu Chuang |
Lisheng Sun |
| HDFS-15338 |
listOpenFiles() should throw InvalidPathException in case of invalid paths |
Minor |
. |
Jinglun |
Jinglun |
| YARN-10160 |
Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo |
Major |
. |
Prabhu Joseph |
Prabhu Joseph |
| HDFS-15255 |
Consider StorageType when DatanodeManager#sortLocatedBlock() |
Major |
. |
Lisheng Sun |
Lisheng Sun |
| YARN-10260 |
Allow transitioning queue from DRAINING to RUNNING state |
Major |
. |
Jonathan Hung |
Bilwa S T |
| HADOOP-17036 |
TestFTPFileSystem failing as ftp server dir already exists |
Minor |
fs, test |
Steve Loughran |
Mikhail Pryakhin |
| HDFS-15356 |
Unify configuration `dfs.ha.allow.stale.reads` to DFSConfigKeys |
Major |
hdfs |
Xiaoqiao He |
Xiaoqiao He |
| HDFS-15358 |
RBF: Unify router datanode UI with namenode datanode UI |
Major |
. |
Ayush Saxena |
Ayush Saxena |
| HADOOP-17042 |
Hadoop distcp throws “ERROR: Tools helper ///usr/lib/hadoop/libexec/tools/hadoop-distcp.sh was not found” |
Minor |
tools/distcp |
Aki Tanaka |
Aki Tanaka |
| HDFS-15202 |
HDFS-client: boost ShortCircuit Cache |
Minor |
dfsclient |
Danil Lipovoy |
Danil Lipovoy |
| HDFS-15207 |
VolumeScanner skip to scan blocks accessed during recent scan peroid |
Minor |
datanode |
Yang Yun |
Yang Yun |
| HDFS-14999 |
Avoid Potential Infinite Loop in DFSNetworkTopology |
Major |
. |
Ayush Saxena |
Ayush Saxena |
| HDFS-13639 |
SlotReleaser is not fast enough |
Major |
hdfs-client |
Gang Xie |
Lisheng Sun |
| HDFS-15369 |
Refactor method VolumeScanner#runLoop() |
Minor |
datanode |
Yang Yun |
Yang Yun |
| HADOOP-14698 |
Make copyFromLocal’s -t option available for put as well |
Major |
. |
Andras Bokor |
Andras Bokor |
| HDFS-10792 |
RedundantEditLogInputStream should log caught exceptions |
Minor |
namenode |
Wei-Chiu Chuang |
Wei-Chiu Chuang |
| YARN-6492 |
Generate queue metrics for each partition |
Major |
capacity scheduler |
Jonathan Hung |
Manikandan R |
| HADOOP-17016 |
Adding Common Counters in ABFS |
Major |
fs/azure |
Mehakmeet Singh |
Mehakmeet Singh |
| HADOOP-16828 |
Zookeeper Delegation Token Manager fetch sequence number by batch |
Major |
. |
Fengnan Li |
Fengnan Li |
| HADOOP-14566 |
Add seek support for SFTP FileSystem |
Minor |
fs |
Azhagu Selvan SP |
Mikhail Pryakhin |
| HADOOP-17047 |
TODO comments exist in trunk while the related issues are already fixed. |
Trivial |
. |
Rungroj Maipradit |
Rungroj Maipradit |
| HADOOP-17020 |
Improve RawFileSystem Performance |
Minor |
fs |
Rajesh Balamohan |
Mehakmeet Singh |
| HDFS-15406 |
Improve the speed of Datanode Block Scan |
Major |
. |
Hemanth Boyina |
Hemanth Boyina |
| HADOOP-17090 |
Increase precommit job timeout from 5 hours to 20 hours |
Major |
build |
Akira Ajisaka |
Akira Ajisaka |
| HADOOP-17084 |
Update Dockerfile_aarch64 to use Bionic |
Major |
build, test |
RuiChen |
zhaorenhai |
| YARN-8047 |
RMWebApp make external class pluggable |
Minor |
. |
Bibin Chundatt |
Bilwa S T |
| YARN-10297 |
TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently |
Major |
. |
Jonathan Hung |
Jim Brennan |
| HADOOP-17127 |
Use RpcMetrics.TIMEUNIT to initialize rpc queueTime and processingTime |
Minor |
common |
Jim Brennan |
Jim Brennan |
| HDFS-15404 |
ShellCommandFencer should expose info about source |
Major |
. |
Chen Liang |
Chen Liang |
| HADOOP-17147 |
Dead link in hadoop-kms/index.md.vm |
Minor |
documentation, kms |
Akira Ajisaka |
Xieming Li |
| HADOOP-17113 |
Adding ReadAhead Counters in ABFS |
Major |
fs/azure |
Mehakmeet Singh |
Mehakmeet Singh |
| YARN-10343 |
Legacy RM UI should include labeled metrics for allocated, total, and reserved resources. |
Major |
. |
Eric Payne |
Eric Payne |
| YARN-1529 |
Add Localization overhead metrics to NM |
Major |
nodemanager |
Gera Shegalov |
Jim Brennan |
| YARN-10361 |
Make custom DAO classes configurable into RMWebApp#JAXBContextResolver |
Major |
. |
Prabhu Joseph |
Bilwa S T |
| YARN-10251 |
Show extended resources on legacy RM UI. |
Major |
. |
Eric Payne |
Eric Payne |
| HDFS-15493 |
Update block map and name cache in parallel while loading fsimage. |
Major |
namenode |
Chengwei Wang |
Chengwei Wang |
| HADOOP-17057 |
ABFS driver enhancement - Allow customizable translation from AAD SPNs and security groups to Linux user and group |
Major |
fs/azure |
Karthik Amarnath |
Karthik Amarnath |
| HADOOP-17194 |
Adding Context class for AbfsClient to pass AbfsConfigurations to limit number of parameters |
Major |
fs/azure |
Mehakmeet Singh |
Mehakmeet Singh |
| HADOOP-17065 |
Adding Network Counters in ABFS |
Major |
fs/azure |
Mehakmeet Singh |
Mehakmeet Singh |
| HADOOP-17159 |
Make UGI support forceful relogin from keytab ignoring the last login time |
Major |
security |
Sandeep Guggilam |
Sandeep Guggilam |
| YARN-10407 |
Add phantomjsdriver.log to gitignore |
Minor |
. |
Takanobu Asanuma |
Takanobu Asanuma |
| YARN-10353 |
Log vcores used and cumulative cpu in containers monitor |
Minor |
yarn |
Jim Brennan |
Jim Brennan |
| YARN-10369 |
Make NMTokenSecretManagerInRM sending NMToken for nodeId DEBUG |
Minor |
yarn |
Jim Brennan |
Jim Brennan |
| YARN-10390 |
LeafQueue: retain user limits cache across assignContainers() calls |
Major |
capacity scheduler, capacityscheduler |
Muhammad Samir Khan |
Muhammad Samir Khan |
| HDFS-15574 |
Remove unnecessary sort of block list in DirectoryScanner |
Major |
. |
Stephen O’Donnell |
Stephen O’Donnell |
| HADOOP-17270 |
Fix testCompressorDecompressorWithExeedBufferLimit to cover the intended scenario |
Minor |
test |
Masatake Iwasaki |
Masatake Iwasaki |
| HDFS-15581 |
Access Controlled HTTPFS Proxy |
Minor |
httpfs |
Richard |
Richard |
| HADOOP-17283 |
Hadoop - Upgrade to JQuery 3.5.1 |
Major |
. |
Aryan Gupta |
Aryan Gupta |
| HADOOP-17267 |
Add debug-level logs in Filesystem#close |
Minor |
fs |
Karen Coppage |
Karen Coppage |
| HADOOP-17284 |
Support BCFKS keystores for Hadoop Credential Provider |
Major |
. |
Xiaoyu Yao |
Xiaoyu Yao |
| HDFS-15415 |
Reduce locking in Datanode DirectoryScanner |
Major |
datanode |
Stephen O’Donnell |
Stephen O’Donnell |
| YARN-10451 |
RM (v1) UI NodesPage can NPE when yarn.io/gpu resource type is defined. |
Major |
. |
Eric Payne |
Eric Payne |
| HADOOP-17021 |
Add concat fs command |
Minor |
fs |
Jinglun |
Jinglun |
| MAPREDUCE-7301 |
Expose Mini MR Cluster attribute for testing |
Minor |
test |
Swaroopa Kadam |
Swaroopa Kadam |
| HDFS-15567 |
[SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. |
Major |
ha, hdfs-client |
Konstantin Shvachko |
Konstantin Shvachko |
| HDFS-15633 |
Avoid redundant RPC calls for getDiskStatus |
Major |
dfsclient |
Ayush Saxena |
Ayush Saxena |
| YARN-10450 |
Add cpu and memory utilization per node and cluster-wide metrics |
Minor |
yarn |
Jim Brennan |
Jim Brennan |
| HADOOP-17302 |
Upgrade to jQuery 3.5.1 in hadoop-sls |
Major |
. |
Aryan Gupta |
Aryan Gupta |
| HDFS-15652 |
Make block size from NNThroughputBenchmark configurable |
Minor |
benchmarks |
Hui Fei |
Hui Fei |
| YARN-10475 |
Scale RM-NM heartbeat interval based on node utilization |
Minor |
yarn |
Jim Brennan |
Jim Brennan |
| HDFS-15665 |
Balancer logging improvement |
Major |
balancer & mover |
Konstantin Shvachko |
Konstantin Shvachko |
| HADOOP-17342 |
Creating a token identifier should not do kerberos name resolution |
Major |
common |
Jim Brennan |
Jim Brennan |
| YARN-10479 |
RMProxy should retry on SocketTimeout Exceptions |
Major |
yarn |
Jim Brennan |
Jim Brennan |
| HDFS-15623 |
Respect configured values of rpc.engine |
Major |
hdfs |
Hector Sandoval Chaverri |
Hector Sandoval Chaverri |
| HADOOP-17369 |
Bump up snappy-java to 1.1.8.1 |
Minor |
. |
Masatake Iwasaki |
Masatake Iwasaki |
| YARN-10480 |
replace href tags with ng-href |
Trivial |
. |
Gabriel Medeiros Coelho |
Gabriel Medeiros Coelho |
| HADOOP-17367 |
Add InetAddress api to ProxyUsers.authorize |
Major |
performance, security |
Ahmed Hussein |
Ahmed Hussein |
| MAPREDUCE-7304 |
Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class |
Major |
mrv2 |
Daniel Fritsi |
Zoltán Erdmann |
| MAPREDUCE-7309 |
Improve performance of reading resource request for mapper/reducers from config |
Major |
applicationmaster |
Wangda Tan |
Peter Bacsko |
| HDFS-15694 |
Avoid calling UpdateHeartBeatState inside DataNodeDescriptor |
Major |
. |
Ahmed Hussein |
Ahmed Hussein |
| HDFS-15703 |
Don’t generate edits for set operations that are no-op |
Major |
namenode |
Ahmed Hussein |
Ahmed Hussein |
| HADOOP-17392 |
Remote exception messages should not include the exception class |
Major |
ipc |
Ahmed Hussein |
Ahmed Hussein |
| HDFS-15706 |
HttpFS: Log more information on request failures |
Major |
httpfs |
Ahmed Hussein |
Ahmed Hussein |
| HADOOP-17389 |
KMS should log full UGI principal |
Major |
. |
Ahmed Hussein |
Ahmed Hussein |
| HADOOP-17425 |
Bump up snappy-java to 1.1.8.2 |
Minor |
. |
L. C. Hsieh |
L. C. Hsieh |
| HDFS-15720 |
namenode audit async logger should add some log4j config |
Minor |
hdfs |
Max Xie |
|
| HDFS-15717 |
Improve fsck logging |
Major |
logging, namenode |
Ahmed Hussein |
Ahmed Hussein |
| HDFS-15704 |
Mitigate lease monitor’s rapid infinite loop |
Major |
namenode |
Ahmed Hussein |
Ahmed Hussein |
| HDFS-15569 |
Speed up the Storage#doRecover during datanode rolling upgrade |
Major |
. |
Hemanth Boyina |
Hemanth Boyina |
| HDFS-15751 |
Add documentation for msync() API to filesystem.md |
Major |
documentation |
Konstantin Shvachko |
Konstantin Shvachko |
| HADOOP-17454 |
[s3a] Disable bucket existence check - set fs.s3a.bucket.probe to 0 |
Major |
. |
Gabor Bota |
Gabor Bota |
| YARN-10538 |
Add recommissioning nodes to the list of updated nodes returned to the AM |
Major |
. |
Srinivas S T |
Srinivas S T |
| YARN-10541 |
capture the performance metrics of ZKRMStateStore |
Minor |
resourcemanager |
Max Xie |
Max Xie |
| HADOOP-17408 |
Optimize NetworkTopology while sorting of block locations |
Major |
common, net |
Ahmed Hussein |
Ahmed Hussein |
| YARN-4589 |
Diagnostics for localization timeouts is lacking |
Major |
. |
Chang Li |
Chang Li |
| YARN-10562 |
Follow up changes for YARN-9833 |
Major |
yarn |
Jim Brennan |
Jim Brennan |
| HDFS-15783 |
Speed up BlockPlacementPolicyRackFaultTolerant#verifyBlockPlacement |
Major |
block placement |
Akira Ajisaka |
Akira Ajisaka |
| YARN-10519 |
Refactor QueueMetricsForCustomResources class to move to yarn-common package |
Major |
. |
Minni Mittal |
Minni Mittal |
| HADOOP-17484 |
Typo in hadop-aws index.md |
Trivial |
documentation, fs/s3 |
Maksim |
Maksim |
| HADOOP-17478 |
Improve the description of hadoop.http.authentication.signature.secret.file |
Minor |
documentation |
Akira Ajisaka |
Akira Ajisaka |
| MAPREDUCE-7317 |
Add latency information in FileOutputCommitter.mergePaths |
Minor |
client |
Jungtaek Lim |
Jungtaek Lim |
| HDFS-15789 |
Lease renewal does not require namesystem lock |
Major |
hdfs |
Jim Brennan |
Jim Brennan |
| HADOOP-17501 |
Fix logging typo in ShutdownHookManager |
Major |
common |
Konstantin Shvachko |
Fengnan Li |
| HADOOP-17354 |
Move Jenkinsfile outside of the root directory |
Major |
build |
Akira Ajisaka |
Akira Ajisaka |
| HADOOP-17508 |
Simplify dependency installation instructions |
Trivial |
documentation |
Gautham Banasandra |
Gautham Banasandra |
| HADOOP-17509 |
Parallelize building of dependencies |
Minor |
build |
Gautham Banasandra |
Gautham Banasandra |
| HDFS-15799 |
Make DisallowedDatanodeException terse |
Minor |
hdfs |
Richard |
Richard |
| HDFS-15813 |
DataStreamer: keep sending heartbeat packets while streaming |
Major |
hdfs |
Jim Brennan |
Jim Brennan |
| MAPREDUCE-7319 |
Log list of mappers at trace level in ShuffleHandler audit log |
Minor |
yarn |
Jim Brennan |
Jim Brennan |
| HADOOP-14402 |
roll out StreamCapabilities across output streams of all filesystems |
Major |
fs, fs/adl, fs/azure, fs/oss, fs/s3, fs/swift |
Steve Loughran |
Steve Loughran |
| HDFS-15821 |
Add metrics for in-service datanodes |
Minor |
. |
Zehao Chen |
Zehao Chen |
| YARN-10626 |
Log resource allocation in NM log at container start time |
Major |
. |
Eric Badger |
Eric Badger |
| HDFS-15815 |
if required storageType are unavailable, log the failed reason during choosing Datanode |
Minor |
block placement |
Yang Yun |
Yang Yun |
| HDFS-15830 |
Support to make dfs.image.parallel.load reconfigurable |
Major |
namenode |
Hui Fei |
Hui Fei |
| HDFS-15835 |
Erasure coding: Add/remove logs for the better readability/debugging |
Minor |
erasure-coding, hdfs |
Bhavik Patel |
Bhavik Patel |
| HDFS-15826 |
Solve the problem of incorrect progress of delegation tokens when loading FsImage |
Major |
. |
JiangHua Zhu |
JiangHua Zhu |
| HDFS-15734 |
[READ] DirectoryScanner#scan need not check StorageType.PROVIDED |
Minor |
datanode |
Yuxuan Wang |
Yuxuan Wang |
| HADOOP-17538 |
Add kms-default.xml and httpfs-default.xml to site index |
Minor |
documentation |
Masatake Iwasaki |
Masatake Iwasaki |
| YARN-10613 |
Config to allow Intra- and Inter-queue preemption to enable/disable conservativeDRF |
Minor |
capacity scheduler, scheduler preemption |
Eric Payne |
Eric Payne |
| YARN-10653 |
Fixed the findbugs issues introduced by YARN-10647. |
Major |
. |
Qi Zhu |
Qi Zhu |
| MAPREDUCE-7324 |
ClientHSSecurityInfo class is in wrong META-INF file |
Major |
. |
Eric Badger |
Eric Badger |
| HADOOP-17546 |
Update Description of hadoop-http-auth-signature-secret in HttpAuthentication.md |
Minor |
. |
Ravuri Sushma sree |
Ravuri Sushma sree |
| YARN-10664 |
Allow parameter expansion in NM_ADMIN_USER_ENV |
Major |
yarn |
Jim Brennan |
Jim Brennan |
| HADOOP-17570 |
Apply YETUS-1102 to re-enable GitHub comments |
Major |
build |
Akira Ajisaka |
Akira Ajisaka |
| HADOOP-17594 |
DistCp: Expose the JobId for applications executing through run method |
Major |
. |
Ayush Saxena |
Ayush Saxena |
| HDFS-15907 |
Reduce Memory Overhead of AclFeature by avoiding AtomicInteger |
Major |
namenode |
Stephen O’Donnell |
Stephen O’Donnell |
| HDFS-15911 |
Provide blocks moved count in Balancer iteration result |
Major |
balancer & mover |
Viraj Jasani |
Viraj Jasani |
| HDFS-15919 |
BlockPoolManager should log stack trace if unable to get Namenode addresses |
Major |
datanode |
Stephen O’Donnell |
Stephen O’Donnell |
| HADOOP-17531 |
DistCp: Reduce memory usage on copying huge directories |
Critical |
. |
Ayush Saxena |
Ayush Saxena |
| HDFS-15879 |
Exclude slow nodes when choose targets for blocks |
Major |
. |
tomscut |
tomscut |
| HADOOP-16870 |
Use spotbugs-maven-plugin instead of findbugs-maven-plugin |
Major |
build |
Akira Ajisaka |
Akira Ajisaka |
| HADOOP-17222 |
Create socket address leveraging URI cache |
Major |
common, hdfs-client |
fanrui |
fanrui |
| HDFS-15932 |
Improve the balancer error message when process exits abnormally. |
Major |
. |
Renukaprasad C |
Renukaprasad C |
| HADOOP-16524 |
Automatic keystore reloading for HttpServer2 |
Major |
. |
Kihwal Lee |
Borislav Iordanov |
| HDFS-15931 |
Fix non-static inner classes for better memory management |
Major |
. |
Viraj Jasani |
Viraj Jasani |
| HADOOP-17371 |
Bump Jetty to the latest version 9.4.35 |
Major |
. |
Wei-Chiu Chuang |
Wei-Chiu Chuang |
| HDFS-15942 |
Increase Quota initialization threads |
Major |
namenode |
Stephen O’Donnell |
Stephen O’Donnell |
| HADOOP-17613 |
Log not flushed fully when daemon shutdown |
Major |
common |
Renukaprasad C |
Renukaprasad C |
| HDFS-15937 |
Reduce memory used during datanode layout upgrade |
Major |
datanode |
Stephen O’Donnell |
Stephen O’Donnell |
| HDFS-15160 |
ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock |
Major |
datanode |
Stephen O’Donnell |
Stephen O’Donnell |
| HADOOP-17569 |
Building native code fails on Fedora 33 |
Major |
build, common |
Kengo Seki |
Masatake Iwasaki |
| HADOOP-17633 |
Please upgrade json-smart dependency to the latest version |
Major |
auth, build |
helen huang |
Viraj Jasani |
| HADOOP-17620 |
DistCp: Use Iterator for listing target directory as well |
Major |
. |
Ayush Saxena |
Ayush Saxena |
| YARN-10743 |
Add a policy for not aggregating for containers which are killed because exceeding container log size limit. |
Major |
. |
Qi Zhu |
Qi Zhu |
| HDFS-15967 |
Improve the log for Short Circuit Local Reads |
Minor |
. |
Bhavik Patel |
Bhavik Patel |
| HADOOP-17675 |
LdapGroupsMapping$LdapSslSocketFactory ClassNotFoundException |
Major |
common |
Tamas Mate |
István Fajth |
| HADOOP-11616 |
Remove workaround for Curator’s ChildReaper requiring Guava 15+ |
Major |
. |
Robert Kanter |
Viraj Jasani |
| HDFS-16003 |
ProcessReport print invalidatedBlocks should judge debug level at first |
Minor |
namanode |
lei w |
lei w |
| HDFS-16007 |
Deserialization of ReplicaState should avoid throwing ArrayIndexOutOfBoundsException |
Major |
. |
junwen yang |
Viraj Jasani |
| HADOOP-17615 |
ADLFS: Update SDK version from 2.3.6 to 2.3.9 |
Minor |
fs/adl |
Bilahari T H |
Bilahari T H |
| HADOOP-16822 |
Provide source artifacts for hadoop-client-api |
Major |
. |
Karel Kolman |
Karel Kolman |
| HADOOP-17680 |
Allow ProtobufRpcEngine to be extensible |
Major |
common |
Hector Sandoval Chaverri |
Hector Sandoval Chaverri |
| YARN-10258 |
Add metrics for ‘ApplicationsRunning’ in NodeManager |
Minor |
nodemanager |
ANANDA G B |
ANANDA G B |
| HDFS-15790 |
Make ProtobufRpcEngineProtos and ProtobufRpcEngineProtos2 Co-Exist |
Critical |
. |
David Mollitor |
Vinayakumar B |