You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 06:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Fri Nov 28, 2025 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3860562610,2chrt481ktimers/319:31:173
361162996055,4cyclictest350747-21kworker/u8:0+events_unbound19:46:081
361162996054,3cyclictest3723740-21kworker/u8:3+events_unbound19:31:261
361162996052,6cyclictest467360-21kworker/u8:0+events_unbound21:16:221
361168995952,4cyclictest585377-21kworker/u8:2+events_unbound23:36:202
361168995852,4cyclictest467360-21kworker/u8:0+events_unbound22:56:262
361168995852,4cyclictest407596-21kworker/u8:2+events_unbound20:01:192
361162995851,5cyclictest622335-21kworker/u8:1+events_unbound00:31:311
361168995751,4cyclictest664244-21kworker/u8:0+flush-8:000:11:282
361162995753,3cyclictest636484-21kworker/u8:0+flush-8:023:51:291
361168995651,4cyclictest622335-21kworker/u8:1+flush-8:000:31:272
361168995651,4cyclictest467360-21kworker/u8:0+flush-8:021:16:252
361168995449,4cyclictest492011-21kworker/u8:2+flush-8:022:06:262
361162995450,3cyclictest622335-21kworker/u8:1+events_unbound00:06:131
361162995446,6cyclictest350747-21kworker/u8:0+events_unbound19:16:241
361162995444,8cyclictest561686-21ntpq22:26:231
361162995443,8cyclictest621232-21ntpq23:26:211
361168995348,3cyclictest622335-21kworker/u8:1+events_unbound23:46:262
361162995349,3cyclictest383146-21kworker/u8:1+flush-8:020:11:281
361162995346,4cyclictest487572-21latency_hist21:15:011
361162995338,11cyclictest282-21systemd-journal22:56:301
361168995246,4cyclictest467360-21kworker/u8:0+flush-8:023:21:232
361162995248,3cyclictest585377-21kworker/u8:2+events_unbound23:36:281
361168995146,4cyclictest636484-21kworker/u8:0+events_unbound23:56:242
361162995147,3cyclictest517229-21kworker/u8:1+events_unbound22:41:141
361162995147,3cyclictest492011-21kworker/u8:2+flush-8:022:06:301
361162995147,3cyclictest492011-21kworker/u8:2+flush-8:021:36:281
361162995147,3cyclictest467360-21kworker/u8:0+events_unbound23:16:111
361162995147,3cyclictest467360-21kworker/u8:0+events_unbound23:16:111
361162995147,3cyclictest3723740-21kworker/u8:3+flush-8:020:51:301
361162995147,3cyclictest350747-21kworker/u8:0+events_unbound20:21:241
361162995145,4cyclictest477409-21kworker/u8:1+events_unbound21:06:141
361168995045,4cyclictest350747-21kworker/u8:0+flush-8:019:51:302
361162995046,3cyclictest602004-21kworker/u8:3+events_unbound00:16:231
361162995046,3cyclictest467360-21kworker/u8:0+flush-8:022:46:271
361162995044,5cyclictest350747-21kworker/u8:0+events_unbound19:26:241
361162995039,8cyclictest406996-21ntpq19:51:231
361155995046,3cyclictest3723740-21kworker/u8:3+flush-8:019:21:310
361162994944,3cyclictest585377-21kworker/u8:2+events_unbound23:11:221
361162994944,3cyclictest585377-21kworker/u8:2+events_unbound23:11:221
361162994940,7cyclictest282-21systemd-journal20:03:221
361168994843,4cyclictest622335-21kworker/u8:1+events_unbound00:36:232
361162994836,8cyclictest282-21systemd-journal20:10:001
361155994843,3cyclictest3723740-21kworker/u8:3+events_unbound19:11:170
361168994741,4cyclictest467360-21kworker/u8:0+flush-8:022:01:222
361168994741,4cyclictest467360-21kworker/u8:0+flush-8:022:01:222
361162994743,3cyclictest492011-21kworker/u8:2+flush-8:022:21:271
361162994741,4cyclictest477409-21kworker/u8:1+events_unbound21:26:251
361162994735,9cyclictest496701-21ntpq21:21:231
361155994737,7cyclictest611327-21ntpq23:16:220
361155994737,7cyclictest611327-21ntpq23:16:220
361162994541,3cyclictest585377-21kworker/u8:2+events_unbound23:31:231
361155994539,4cyclictest3723740-21kworker/u8:3+flush-8:020:46:260
361155994538,5cyclictest632625-21diskstats23:41:120
361155994536,7cyclictest576733-21ntpq22:41:210
361155994536,6cyclictest383146-21kworker/u8:1+flush-8:020:11:210
361162994440,3cyclictest585377-21kworker/u8:2+events_unbound23:06:211
361162994440,3cyclictest492011-21kworker/u8:2+flush-8:022:16:261
361155994439,3cyclictest350747-21kworker/u8:0+flush-8:019:20:010
361162994339,3cyclictest676368-21kworker/u8:3+events_unbound00:26:281
361162994339,3cyclictest517229-21kworker/u8:1+flush-8:022:31:131
361162994338,4cyclictest622335-21kworker/u8:1+flush-8:000:01:161
361168994237,4cyclictest592110-21kworker/u8:1+flush-8:023:01:222
361162994238,3cyclictest567152-21kworker/u8:2+events_unbound22:36:231
361162994238,3cyclictest517229-21kworker/u8:1+events_unbound22:11:201
361162994137,3cyclictest622335-21kworker/u8:1+flush-8:000:21:241
361162994136,4cyclictest350747-21kworker/u8:0+events_unbound20:26:211
361162994130,8cyclictest397029-21ntpq19:41:231
361162994126,11cyclictest282-21systemd-journal23:23:311
361162994036,3cyclictest622335-21kworker/u8:1+flush-8:023:41:131
361162994036,3cyclictest3723740-21kworker/u8:3+flush-8:019:11:281
361162994036,3cyclictest350747-21kworker/u8:0+events_unbound19:56:241
361162994035,4cyclictest636484-21kworker/u8:0+events_unbound23:56:301
361162993935,3cyclictest517229-21kworker/u8:1+flush-8:021:51:181
361155993935,3cyclictest585377-21kworker/u8:2+flush-8:023:31:260
361155993935,3cyclictest383146-21kworker/u8:1+flush-8:020:06:260
361162993834,3cyclictest467360-21kworker/u8:0+flush-8:021:41:121
361162993834,3cyclictest383146-21kworker/u8:1+events_unbound20:41:141
361162993834,3cyclictest383146-21kworker/u8:1+events_unbound20:36:281
361162993833,4cyclictest350747-21kworker/u8:0+events_unbound19:21:131
361155993831,5cyclictest382042-21ntpq19:26:200
361155993829,7cyclictest282-21systemd-journal22:10:000
3562092384,9sleep10-21swapper/119:05:191
361168993729,5cyclictest477409-21kworker/u8:1+events_unbound21:26:132
361162993733,3cyclictest517229-21kworker/u8:1+flush-8:021:46:121
361162993733,3cyclictest383146-21kworker/u8:1+flush-8:020:16:221
361162993730,5cyclictest492011-21kworker/u8:2+events_unbound21:35:001
361162993632,3cyclictest492011-21kworker/u8:2+flush-8:021:56:311
361162993632,3cyclictest492011-21kworker/u8:2+flush-8:021:56:301
361162993632,3cyclictest3723740-21kworker/u8:3+events_unbound20:48:331
361155993631,3cyclictest3723740-21kworker/u8:3+flush-8:020:51:020
361155993630,4cyclictest622335-21kworker/u8:1+events_unbound00:21:260
361155993630,4cyclictest477409-21kworker/u8:1+events_unbound21:21:210
36116899358,13cyclictest569892-21cat22:36:182
361168993528,5cyclictest413023-21cat20:00:012
361162993530,4cyclictest447527-21kworker/u8:2+flush-8:020:56:121
361155993530,3cyclictest517229-21kworker/u8:1+flush-8:022:36:180
361155993529,4cyclictest350747-21kworker/u8:0+flush-8:019:51:150
361155993528,5cyclictest517229-21kworker/u8:1+events_unbound22:46:280
36116899347,16cyclictest426399-21cut20:11:182
361162993430,3cyclictest676368-21kworker/u8:3+events_unbound00:36:281
361162993430,3cyclictest467360-21kworker/u8:0+events_unbound22:01:091
361162993430,3cyclictest467360-21kworker/u8:0+events_unbound22:01:081
361162993430,3cyclictest383146-21kworker/u8:1+events_unbound20:31:101
361162993429,4cyclictest467360-21kworker/u8:0+events_unbound22:50:071
361155993430,3cyclictest652069-21kworker/u8:4+flush-8:000:06:260
361155993430,3cyclictest467360-21kworker/u8:0+flush-8:021:56:280
361155993430,3cyclictest467360-21kworker/u8:0+flush-8:021:56:280
361155993428,4cyclictest467360-21kworker/u8:0+flush-8:022:26:270
361155993427,5cyclictest492011-21kworker/u8:2+events_unbound22:16:210
361155993427,5cyclictest3723740-21kworker/u8:3+flush-8:021:11:230
361155993425,6cyclictest282-21systemd-journal20:00:010
361155993422,10cyclictest282-21systemd-journal20:31:280
36115599340,32cyclictest1-21systemd19:40:010
361168993329,3cyclictest652069-21kworker/u8:4+events_unbound00:01:292
361168993328,4cyclictest567152-21kworker/u8:2+flush-8:022:32:262
361168993326,5cyclictest468120-21apt-get20:56:072
361162993326,5cyclictest641950-21latency_hist23:50:011
361155993328,4cyclictest477409-21kworker/u8:1+flush-8:021:09:110
361155993324,2cyclictest651836-21sendmail00:00:000
361162993227,4cyclictest350747-21kworker/u8:0+events_unbound19:36:291
361155993228,3cyclictest517229-21kworker/u8:1+flush-8:022:11:280
361155993226,4cyclictest467360-21kworker/u8:0+events_unbound21:36:210
361155993225,5cyclictest572900-21kworker/u8:3+events_unbound22:51:250
361155993225,5cyclictest350747-21kworker/u8:0+events_unbound19:46:190
361155993222,7cyclictest350747-21kworker/u8:0+flush-8:019:41:200
361168993126,3cyclictest361172-21latency_hist19:10:012
361162993127,3cyclictest622335-21kworker/u8:1+events_unbound00:12:321
361155993126,4cyclictest585377-21kworker/u8:2+events_unbound23:11:240
361155993126,4cyclictest585377-21kworker/u8:2+events_unbound23:11:240
361155993126,4cyclictest3723740-21kworker/u8:3+events_unbound21:00:010
361155993125,4cyclictest585377-21kworker/u8:2+flush-8:023:06:120
361162993026,3cyclictest3723740-21kworker/u8:3+flush-8:021:01:411
361155993027,2cyclictest517169-21unixbench_singl21:41:280
361155993025,4cyclictest676368-21kworker/u8:3+events_unbound00:36:270
361155993025,3cyclictest502117-21kworker/u8:3+flush-8:021:31:180
361155993024,4cyclictest602004-21kworker/u8:3+flush-8:023:51:230
361155993023,5cyclictest492011-21kworker/u8:2+flush-8:022:21:220
361168992925,2cyclictest437553-21unixbench_singl20:21:292
361155992924,4cyclictest383146-21kworker/u8:1+flush-8:020:41:440
361155992924,4cyclictest350747-21kworker/u8:0+events_unbound20:21:220
361155992923,4cyclictest641619-21unixbench_singl23:46:290
361155992923,4cyclictest585377-21kworker/u8:2+events_unbound23:05:000
361155992920,7cyclictest677221-21diskstats00:26:140
361168992825,2cyclictest543334-21fschecks_time22:11:122
361168992824,3cyclictest383146-21kworker/u8:1+events_unbound19:34:442
361168992824,2cyclictest373649-21fschecks_time19:21:152
361168992823,4cyclictest676368-21kworker/u8:3+flush-8:000:24:252
361162992825,2cyclictest467360-21kworker/u8:0+events_unbound23:01:291
36115599282,25cyclictest427986-21kworker/0:1+events23:20:100
361155992822,4cyclictest527386-21cron21:55:000
361155992822,4cyclictest517229-21kworker/u8:1+events_unbound21:46:120
361168992724,2cyclictest622788-21fschecks_time23:31:112
361168992724,2cyclictest378530-21fschecks_count19:26:122
361168992723,3cyclictest392421-21unixbench_multi19:36:292
361168992722,4cyclictest572900-21kworker/u8:3+events_unbound22:44:002
361168992722,3cyclictest657369-21cpuspeed_turbos00:06:092
361168992719,6cyclictest521548-21needreboot21:46:202
361155992723,3cyclictest447527-21kworker/u8:2+events_unbound21:01:310
361155992723,3cyclictest3723740-21kworker/u8:3+flush-8:020:38:010
361155992722,4cyclictest622335-21kworker/u8:1+events_unbound00:31:290
361155992722,4cyclictest432582-21unixbench_singl20:16:280
36115599272,24cyclictest427986-21kworker/0:1+events22:31:010
36115599272,24cyclictest427986-21kworker/0:1+events22:04:460
36115599272,24cyclictest427986-21kworker/0:1+events22:04:460
361155992722,3cyclictest442813-21systemd-run20:30:000
361155992722,3cyclictest387541-21unixbench_singl19:31:270
36115599271,25cyclictest427986-21kworker/0:1+events22:55:500
361168992622,3cyclictest607233-21turbostat.cron23:15:002
361168992622,3cyclictest607233-21turbostat.cron23:14:592
361168992620,4cyclictest485124-21kthreadcore21:11:182
36116899260,7cyclictest433078-21sed20:20:012
361155992622,3cyclictest467360-21kworker/u8:0+flush-8:021:16:270
361155992621,3cyclictest602004-21kworker/u8:3+flush-8:000:16:240
361155992620,4cyclictest467360-21kworker/u8:0+flush-8:023:26:230
361155992619,5cyclictest350747-21kworker/u8:0+flush-8:019:56:220
36116899253,2cyclictest421rcuog/221:01:302
36116899252,4cyclictest0-21swapper/220:36:172
361168992521,3cyclictest372973-21cron19:20:002
361168992521,2cyclictest398433-21aten2_r5power_p19:46:082
361168992515,7cyclictest512383-21cat21:40:002
361155992521,3cyclictest652069-21kworker/u8:4+flush-8:000:01:310
361168992420,2cyclictest462834-21latency_hist20:50:002
36115599245,18cyclictest427986-21kworker/0:1+events21:26:540
361155992417,5cyclictest282-21systemd-journal23:39:090
361155992414,8cyclictest282-21systemd-journal00:15:000
3562082243,4sleep00-21swapper/019:05:180
361168992319,3cyclictest585377-21kworker/u8:2+events_unbound23:15:072
361168992319,3cyclictest585377-21kworker/u8:2+events_unbound23:15:072
361168992318,4cyclictest350747-21kworker/u8:0+flush-8:020:26:422
361168992313,8cyclictest552608-21awk22:20:002
36116899222,9cyclictest0-21swapper/221:56:262
36116899222,9cyclictest0-21swapper/221:56:262
36116899222,4cyclictest0-21swapper/200:26:142
361168992219,2cyclictest512916-21aten2_r5power_c21:41:082
361168992217,4cyclictest636484-21kworker/u8:0+events_unbound23:54:182
361168992211,8cyclictest507340-21awk21:35:002
36116899220,11cyclictest601113-21tr23:06:192
36116899213,8cyclictest0-21swapper/222:20:002
36116899213,8cyclictest0-21swapper/220:41:162
36116899212,4cyclictest0-21swapper/222:46:282
361168992117,3cyclictest517229-21kworker/u8:1+events_unbound22:26:452
361168992117,2cyclictest0-21swapper/223:41:212
361168992115,1cyclictest421rcuog/200:16:182
36116899210,4cyclictest393309-21cpu19:41:092
36116899203,6cyclictest0-21swapper/223:26:132
36116899202,6cyclictest0-21swapper/221:21:092
36116899202,6cyclictest0-21swapper/219:12:122
36116899202,4cyclictest0-21swapper/221:54:082
36116899202,4cyclictest0-21swapper/220:09:432
361168992012,6cyclictest478029-21cut21:06:122
36116899200,11cyclictest443128-21seq20:31:102
36116899192,8cyclictest0-21swapper/220:51:242
36116899192,4cyclictest0-21swapper/222:53:122
3562332183,8sleep30-21swapper/319:05:373
5277352162,9sleep30-21swapper/321:56:093
5277352162,9sleep30-21swapper/321:56:083
6238162145,5sleep30-21swapper/323:31:163
5679912142,7sleep30-21swapper/322:36:103
5372712142,7sleep30-21swapper/322:05:003
5372712142,7sleep30-21swapper/322:05:003
6608232132,7sleep30-21swapper/300:06:193
6023022132,6sleep30-21swapper/323:10:013
6023022132,6sleep30-21swapper/323:10:013
5918722132,7sleep30-21swapper/322:56:293
5776032132,6sleep30-21swapper/322:45:013
5122832132,7sleep30-21swapper/321:40:003
6818792122,6sleep30-21swapper/300:31:113
6766542122,6sleep30-21swapper/300:25:013
6721512122,6sleep30-21swapper/300:21:123
6679362123,5sleep30-21swapper/300:16:193
6630212122,6sleep30-21swapper/300:11:173
6517062122,6sleep30-21swapper/323:59:013
6431732122,6sleep30-21swapper/323:51:193
6378352122,6sleep30-21swapper/323:46:173
6328162122,6sleep30-21swapper/323:41:153
6113392122,6sleep30-21swapper/323:16:233
6113392122,6sleep30-21swapper/323:16:233
5981892122,6sleep30-21swapper/323:06:163
5932122122,6sleep30-21swapper/323:01:143
5827802122,6sleep30-21swapper/322:50:313
5668272122,6sleep30-21swapper/322:31:293
5176112122,6sleep30-21swapper/321:45:023
4829712122,6sleep30-21swapper/321:11:103
4769622122,6sleep30-21swapper/321:01:313
4598172122,6sleep30-21swapper/320:46:183
4428052122,6sleep30-21swapper/320:30:013
4335242122,6sleep30-21swapper/320:21:143
4186592122,6sleep30-21swapper/320:06:133
4132952122,6sleep30-21swapper/320:00:033
4036882122,6sleep30-21swapper/319:51:173
3935802122,6sleep30-21swapper/319:41:133
3927932122,6sleep30-21swapper/319:40:013
3771542122,6sleep30-21swapper/319:21:273
3726772122,6sleep30-21swapper/319:16:293
6905632112,5sleep30-21swapper/300:36:213
6526272112,6sleep30-21swapper/300:01:183
6319482112,6sleep30-21swapper/323:39:143
6212412112,5sleep30-21swapper/323:26:223
6162012112,5sleep30-21swapper/323:21:213
5765732112,5sleep30-21swapper/322:41:193
5582822112,6sleep30-21swapper/322:26:153
5532452112,5sleep30-21swapper/322:21:133
5481972112,5sleep30-21swapper/322:16:123
5452302111,6sleep30-21swapper/322:11:183
5382302112,5sleep30-21swapper/322:06:163
5231162112,5sleep30-21swapper/321:51:163
5169912112,5sleep30-21swapper/321:41:283
5029562112,5sleep30-21swapper/321:31:133
4925492112,5sleep30-21swapper/321:20:013
4921042112,5sleep30-21swapper/321:16:283
4784152112,5sleep30-21swapper/321:06:173
4678102112,5sleep30-21swapper/320:55:003
4635432112,5sleep30-21swapper/320:51:163
4532032112,5sleep30-21swapper/320:40:173
4480082112,5sleep30-21swapper/320:35:313
4289942112,5sleep30-21swapper/320:16:173
4232872114,4sleep30-21swapper/320:10:113
4125312112,5sleep30-21swapper/319:56:303
3982602112,6sleep30-21swapper/319:45:193
3785112112,5sleep30-21swapper/319:26:123
3660792112,6sleep30-21swapper/319:14:373
4976942102,5sleep30-21swapper/321:25:033
4381572102,5sleep30-21swapper/320:26:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional