Knowledge Base - Solution

Vision db no data

Wednesday, July 29, 2020
Submitted by ryan.grace on July 8, 2021 - 12:46pm
Problem: 
Vision no data being consumed at ingest df -k showed /tmp partition full on investigation full with hs_err, these are java dumps due to kafka process crashing. also in /var/log/messages we see Mar 19 09:30:17 db1 kafka-server-start.sh: 2020-03-19 09:30:17,173 WARN KAFKA.Log: Found a corrupted index file due to requirement failed: Corrupt index found, index file (/bigdee/opt/data/kafka/logs/kafka_lte_segments-1/00000000005655889921.index) has non-zero size but the last offset is 5655889921 which is no larger than the base offset5655889921.}. deleting /bigdee/opt/data/kafka/logs/kafka_lte_segments-1/00000000005655889921.timeindex, /bigdee/opt/data/kafka/logs/kafka_lte_segments-1/00000000005655889921.index, and /bigdee/opt/data/kafka/logs/kafka_lte_segments-1/00000000005655889921.txnindex and rebuilding index... Mar 19 09:30:17 db1 kafka-server-start.sh: 2020-03-19 09:30:17,194 WARN KAFKA.Log: Found a corrupted index file due to requirement failed: Corrupt index found, index file (/bigdee/opt/data/kafka/logs/__consumer_offsets-37/00000000000000000388.index) has non-zero size but the last offset is 388 which is no larger than the base offset 388.}. deleting /bigdee/opt/data/kafka/logs/__consumer_offsets-37/00000000000000000388.timeindex, /bigdee/opt/data/kafka/logs/__consumer_offsets-37/00000000000000000388.index, and /bigdee/opt/data/kafka/logs/__consumer_offsets-37/00000000000000000388.txnindex and rebuilding index... Mar 19 09:30:17 db1 kafka-server-start.sh: 2020-03-19 09:30:17,212 WARN KAFKA.Log: Found a corrupted index file due to requirement failed: Corrupt index found, index file (/bigdee/opt/data/kafka/logs/kafka_umts_segments-5/00000000001474807281.index) has non-zero size but the last offset is 1474807281 which is no larger than the base offset 1474807281.}. deleting /bigdee/opt/data/kafka/logs/kafka_umts_segments-5/00000000001474807281.timeindex, /bigdee/opt/data/kafka/logs/kafka_umts_segments-5/00000000001474807281.index, and /bigdee/opt/data/kafka/logs/kafka_umts_segments-5/00000000001474807281.txnindex and rebuilding index...
Solution: 
go to the vision cluster with the depot then run the repair on bigdata cd /visionreleases/Vision//ansible sh deployer.sh "bigdata" -e repair=true

Product Family:

Support at Every Step

We provide support, services, comprehensive training and the resources you need. It’s all part of what we do to maximize the value of your VIAVI investment.

Value add services that compliment your VIAVI system solution and instrument portfolio to provide a total cost of ownership

Customer Service that provides Return Material Authorization (RMA) for repair and calibration

Technical education solutions, product training, and blended learning for technicians who are using new products or working with existing tools

Technical Assistance Center to assist you in using/configuring products or address issues regarding product performance