You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 20:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Tue Nov 18, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
60421440,10sleep131899cyclictest07:02:501
3129910336,61cyclictest0-21swapper/010:38:000
243799980,5rtkit-daemon0-21swapper/207:56:432
31299923,43cyclictest11666-21cpuspeed_turbos10:47:400
31299923,43cyclictest11666-21cpuspeed_turbos10:47:400
320999022,9cyclictest26888-21telnet12:27:383
312998932,11cyclictest5677-21tr11:42:400
32099863,41cyclictest729-21cpuspeed_turbos11:32:393
32099863,41cyclictest729-21cpuspeed_turbos11:32:393
312998623,13cyclictest0-21swapper/009:02:300
243799860,5rtkit-daemon2436-21rtkit-daemon08:08:051
243799850,6rtkit-daemon0-21swapper/007:34:062
32099844,36cyclictest16523-21tr08:42:463
312998433,8cyclictest1243-21snmpd09:16:270
312998429,8cyclictest17276-21munin-node12:07:570
243799840,8rtkit-daemon0-21swapper/207:03:003
320998326,14cyclictest0-21swapper/308:02:323
320998326,14cyclictest0-21swapper/308:02:323
320998231,9cyclictest23733-21timerwakeupswit10:03:023
243799780,8rtkit-daemon0-21swapper/207:52:513
320997428,10cyclictest0-21swapper/307:02:353
320997427,10cyclictest0-21swapper/308:42:043
320997427,10cyclictest0-21swapper/308:42:043
243799700,6rtkit-daemon0-21swapper/108:07:570
320996927,10cyclictest0-21swapper/310:22:393
183952682,9sleep20-21swapper/212:07:502
227872674,11sleep20-21swapper/207:47:512
31999653,36cyclictest0-21swapper/207:42:492
31999635,32cyclictest0-21swapper/210:17:402
31999635,31cyclictest0-21swapper/207:18:062
319996322,14cyclictest0-21swapper/209:32:562
318996343,14cyclictest0-21swapper/107:17:471
31299633,52cyclictest0-21swapper/010:26:310
31999624,39cyclictest0-21swapper/210:08:012
31999624,39cyclictest0-21swapper/210:08:012
31999624,31cyclictest0-21swapper/210:43:002
31999619,23cyclictest10097-21munin-node11:52:442
31999614,31cyclictest0-21swapper/210:33:052
31999614,30cyclictest0-21swapper/210:52:522
318996143,12cyclictest0-21swapper/111:32:361
318996143,12cyclictest0-21swapper/111:32:361
31999605,30cyclictest0-21swapper/209:47:562
31999605,14cyclictest0-21swapper/211:46:532
31999605,14cyclictest0-21swapper/210:13:012
31999604,31cyclictest0-21swapper/207:37:572
319996022,13cyclictest0-21swapper/211:27:382
319996013,38cyclictest0-21swapper/210:32:252
318996040,13cyclictest0-21swapper/112:02:441
31999595,47cyclictest0-21swapper/211:47:472
31999595,47cyclictest0-21swapper/208:47:462
31999595,28cyclictest1369-21runrttasks12:17:342
31999595,28cyclictest1369-21runrttasks12:17:342
31999595,28cyclictest0-21swapper/209:02:422
31999594,32cyclictest0-21swapper/209:37:582
31999594,31cyclictest0-21swapper/209:08:022
31999593,31cyclictest0-21swapper/209:42:402
31999585,29cyclictest0-21swapper/209:17:402
31999585,29cyclictest0-21swapper/209:17:402
31999584,29cyclictest0-21swapper/212:27:532
31999584,29cyclictest0-21swapper/212:12:522
31999584,29cyclictest0-21swapper/211:32:422
31999584,29cyclictest0-21swapper/211:32:422
31999584,29cyclictest0-21swapper/207:27:472
319995820,13cyclictest0-21swapper/211:07:472
318995823,14cyclictest0-21swapper/107:07:491
243799580,5rtkit-daemon2436-21rtkit-daemon07:22:520
32099574,32cyclictest0-21swapper/311:07:373
320995731,9cyclictest3288-21sendmail_mailqu08:13:013
31999575,43cyclictest0-21swapper/208:37:522
31999575,43cyclictest0-21swapper/208:37:522
31999575,28cyclictest0-21swapper/211:41:522
31999575,28cyclictest0-21swapper/211:17:432
31999575,26cyclictest0-21swapper/209:26:292
31999575,26cyclictest0-21swapper/209:26:292
31999574,29cyclictest0-21swapper/211:22:412
31999574,28cyclictest0-21swapper/210:57:402
31299574,46cyclictest0-21swapper/009:37:280
31999564,29cyclictest0-21swapper/209:53:012
318995639,12cyclictest0-21swapper/112:07:551
318995638,12cyclictest0-21swapper/109:32:471
318995637,13cyclictest0-21swapper/111:12:381
31999555,33cyclictest0-21swapper/208:32:592
31999555,27cyclictest0-21swapper/210:51:432
31999555,27cyclictest0-21swapper/210:51:432
31999555,26cyclictest0-21swapper/208:11:042
31999554,27cyclictest0-21swapper/212:02:232
31999554,24cyclictest18514-21munin-node11:02:422
31999553,45cyclictest26992-21munin-node07:57:512
31999553,45cyclictest26992-21munin-node07:57:512
31999552,46cyclictest0-21swapper/208:55:402
318995539,11cyclictest0-21swapper/109:47:581
318995538,11cyclictest0-21swapper/111:47:531
318995536,13cyclictest0-21swapper/107:27:461
169352554,12sleep10-21swapper/107:34:191
50102544,10sleep20-21swapper/207:09:292
320995430,11cyclictest0-21swapper/310:57:423
31999545,32cyclictest0-21swapper/209:27:472
31999545,31cyclictest0-21swapper/210:22:462
319995420,14cyclictest0-21swapper/210:39:592
319995420,14cyclictest0-21swapper/209:16:272
318995439,10cyclictest0-21swapper/109:32:291
318995437,12cyclictest0-21swapper/110:17:571
318995436,12cyclictest0-21swapper/107:27:341
318995420,14cyclictest0-21swapper/112:22:221
318995420,14cyclictest0-21swapper/112:22:221
312995422,14cyclictest0-21swapper/007:42:320
243799540,6rtkit-daemon2436-21rtkit-daemon12:13:030
243799540,5rtkit-daemon2436-21rtkit-daemon11:17:430
243799540,5rtkit-daemon2436-21rtkit-daemon10:57:510
243799540,5rtkit-daemon2436-21rtkit-daemon10:52:430
243799540,5rtkit-daemon2436-21rtkit-daemon07:12:520
60652533,8sleep16085-21fschecks_count07:12:511
320995331,10cyclictest0-21swapper/307:32:433
32099533,23cyclictest0-21swapper/311:44:423
320995322,13cyclictest0-21swapper/307:28:353
31999535,32cyclictest0-21swapper/208:22:452
319995335,11cyclictest4586-21cpuspeed_turbos08:17:482
319995322,16cyclictest0-21swapper/207:02:532
31899535,30cyclictest0-21swapper/111:37:581
31899535,29cyclictest0-21swapper/109:47:281
318995336,11cyclictest0-21swapper/110:47:551
318995336,11cyclictest0-21swapper/110:47:551
318995322,14cyclictest0-21swapper/108:13:001
318995321,14cyclictest0-21swapper/111:46:561
318995319,15cyclictest0-21swapper/111:57:521
31299533,44cyclictest0-21swapper/010:17:560
31299533,43cyclictest0-21swapper/007:48:040
243799530,5rtkit-daemon2436-21rtkit-daemon12:27:400
243799530,5rtkit-daemon2436-21rtkit-daemon11:57:560
243799530,5rtkit-daemon2436-21rtkit-daemon11:52:410
243799530,5rtkit-daemon2436-21rtkit-daemon11:37:570
243799530,5rtkit-daemon2436-21rtkit-daemon10:37:070
243799530,5rtkit-daemon2436-21rtkit-daemon10:27:590
243799530,5rtkit-daemon2436-21rtkit-daemon07:11:300
194532532,7sleep01243-21snmpd09:56:440
32099523,43cyclictest0-21swapper/310:42:143
31999524,32cyclictest0-21swapper/208:12:552
31999524,31cyclictest0-21swapper/212:02:542
319995234,10cyclictest29699-21cpuspeed_turbos08:02:482
319995234,10cyclictest24999-21cat08:58:042
31999523,43cyclictest19753-21munin-node09:57:522
31899525,40cyclictest0-21swapper/112:22:391
31899525,29cyclictest0-21swapper/110:16:371
31899525,29cyclictest0-21swapper/109:22:541
31899525,29cyclictest0-21swapper/109:22:541
31899524,31cyclictest0-21swapper/109:52:461
31899524,30cyclictest0-21swapper/110:22:531
31899524,24cyclictest0-21swapper/111:02:451
318995235,11cyclictest0-21swapper/110:52:421
318995232,12cyclictest0-21swapper/110:02:461
31899523,25cyclictest0-21swapper/107:42:491
312995224,13cyclictest0-21swapper/008:52:300
280142523,42sleep10-21swapper/107:57:581
280142523,42sleep10-21swapper/107:57:581
266302523,43sleep10-21swapper/107:56:041
249602523,42sleep10-21swapper/110:07:451
249602523,42sleep10-21swapper/110:07:451
243799520,5rtkit-daemon2436-21rtkit-daemon11:32:490
243799520,5rtkit-daemon2436-21rtkit-daemon11:32:490
243799520,5rtkit-daemon2436-21rtkit-daemon11:12:500
243799520,5rtkit-daemon2436-21rtkit-daemon07:17:590
243799520,4rtkit-daemon2436-21rtkit-daemon10:42:590
320995129,10cyclictest0-21swapper/308:17:463
31999514,32cyclictest0-21swapper/208:42:532
31899515,28cyclictest0-21swapper/109:07:451
31899515,28cyclictest0-21swapper/108:28:021
31899514,29cyclictest0-21swapper/109:02:311
31899514,29cyclictest0-21swapper/108:43:011
31899514,28cyclictest0-21swapper/108:52:421
318995136,11cyclictest0-21swapper/111:27:451
318995132,13cyclictest0-21swapper/110:37:501
31899513,25cyclictest0-21swapper/110:33:001
318995122,13cyclictest0-21swapper/108:57:431
31899512,10cyclictest0-21swapper/108:03:081
318995114,12cyclictest0-21swapper/108:48:021
31299515,39cyclictest0-21swapper/009:42:420
31299513,42cyclictest28290-21basename09:07:440
312995122,14cyclictest0-21swapper/007:27:460
312995121,14cyclictest0-21swapper/007:56:420
291162517,38sleep00-21swapper/008:02:200
291162517,38sleep00-21swapper/008:02:200
243799510,5rtkit-daemon2436-21rtkit-daemon12:02:480
243799510,5rtkit-daemon2436-21rtkit-daemon11:27:570
243799510,5rtkit-daemon2436-21rtkit-daemon11:07:430
242582514,41sleep10-21swapper/107:51:291
210602514,11sleep10-21swapper/111:07:391
52182505,39sleep30-21swapper/307:12:123
320995029,10cyclictest0-21swapper/309:21:283
320995029,10cyclictest0-21swapper/309:21:283
320995026,10cyclictest0-21swapper/311:27:543
320995026,10cyclictest0-21swapper/307:47:503
320995023,11cyclictest23422-21aten2.4-expect08:57:433
320995022,12cyclictest0-21swapper/310:16:483
320995021,13cyclictest0-21swapper/309:32:503
319995033,11cyclictest23901-21grep10:03:032
31899505,26cyclictest0-21swapper/109:21:281
31899505,26cyclictest0-21swapper/109:21:281
31899504,28cyclictest0-21swapper/112:27:591
31899504,28cyclictest0-21swapper/109:59:131
31899504,24cyclictest0-21swapper/112:12:361
31899503,24cyclictest0-21swapper/107:37:591
312995035,9cyclictest25285-21sed12:22:460
312995022,12cyclictest0-21swapper/008:27:560
243799500,5rtkit-daemon2436-21rtkit-daemon11:47:410
209462505,38sleep00-21swapper/008:52:350
181822503,41sleep10-21swapper/111:01:141
13422503,40sleep01435-21gltestperf07:02:580
96082494,37sleep30-21swapper/308:27:513
90162493,39sleep10-21swapper/110:42:401
325502494,6sleep00-21swapper/007:00:010
32099493,28cyclictest0-21swapper/308:32:513
320994928,10cyclictest0-21swapper/308:06:173
319994932,10cyclictest9949-21hwlatdetect08:27:532
31899496,10cyclictest0-21swapper/109:12:541
31899495,26cyclictest0-21swapper/111:22:341
31899495,18cyclictest10251-21aten2.4-expect11:52:371
31899494,30cyclictest0-21swapper/108:22:431
31899494,27cyclictest0-21swapper/109:37:441
318994918,14cyclictest0-21swapper/111:17:411
318994918,13cyclictest0-21swapper/108:17:551
312994929,8cyclictest1243-21snmpd12:21:580
312994929,8cyclictest1243-21snmpd12:21:580
312994923,11cyclictest0-21swapper/009:57:560
312994922,12cyclictest0-21swapper/010:02:460
87192482,40sleep00-21swapper/008:26:430
320994831,11cyclictest25729-21needreboot10:07:543
320994831,11cyclictest25729-21needreboot10:07:543
320994831,11cyclictest0-21swapper/312:21:553
320994831,11cyclictest0-21swapper/312:21:553
320994831,10cyclictest12846-21cpuspeed_turbos11:57:393
320994826,10cyclictest0-21swapper/312:07:373
320994825,9cyclictest4183-21munin-node10:32:453
320994825,11cyclictest0-21swapper/307:42:463
320994824,10cyclictest0-21swapper/309:07:493
320994820,23cyclictest13056-21df_abs09:42:433
320994819,13cyclictest0-21swapper/310:42:583
318994832,10cyclictest0-21swapper/108:32:441
318994817,14cyclictest0-21swapper/110:27:391
312994829,8cyclictest24555-21telnet10:07:400
312994829,8cyclictest24555-21telnet10:07:400
204792482,40sleep00-21swapper/011:05:040
320994725,8cyclictest25770-21munin-node09:02:483
320994725,10cyclictest0-21swapper/309:52:413
320994725,10cyclictest0-21swapper/307:17:493
320994724,11cyclictest0-21swapper/310:52:383
319994729,11cyclictest5611-21aten2.4-expect07:12:452
319994726,15cyclictest0-21swapper/207:22:462
319994726,10cyclictest23617-21tr11:12:402
318994716,14cyclictest0-21swapper/108:37:481
318994716,14cyclictest0-21swapper/108:37:481
31299474,28cyclictest0-21swapper/008:17:510
31299473,39cyclictest0-21swapper/009:32:290
312994728,8cyclictest1015-21crond07:47:320
312994720,11cyclictest27918-21latency_hist09:07:300
156042472,39sleep00-21swapper/008:38:080
156042472,39sleep00-21swapper/008:38:080
32099464,35cyclictest0-21swapper/309:12:433
32099464,13cyclictest0-21swapper/309:27:513
320994631,9cyclictest20010-21tr12:12:383
320994628,11cyclictest5789-21cpuspeed_turbos07:12:473
320994627,12cyclictest0-21swapper/309:27:183
320994627,12cyclictest0-21swapper/309:27:183
31299463,36cyclictest0-21swapper/009:17:440
31299463,36cyclictest0-21swapper/009:17:440
312994628,6cyclictest18337-21latency_hist08:47:310
312994626,7cyclictest17109-21latency_hist09:52:280
312994623,8cyclictest30213-21latency_hist11:27:240
312994623,10cyclictest0-21swapper/009:22:450
312994623,10cyclictest0-21swapper/009:22:450
108322462,37sleep00-21swapper/009:37:490
32099455,24cyclictest688-21processes08:08:033
32099454,35cyclictest0-21swapper/311:56:363
32099454,33cyclictest0-21swapper/310:47:573
32099454,33cyclictest0-21swapper/310:47:573
320994531,8cyclictest1369-21runrttasks11:22:273
32099453,36cyclictest0-21swapper/312:26:033
32099453,35cyclictest0-21swapper/311:37:443
320994523,9cyclictest23286-21munin-node11:12:513
320994523,9cyclictest0-21swapper/310:17:403
320994523,11cyclictest0-21swapper/307:39:083
320994521,11cyclictest0-21swapper/308:48:013
320994518,10cyclictest8734-21irqstats11:47:473
31999454,13cyclictest0-21swapper/207:02:352
312994523,10cyclictest1909-21telnet08:12:440
312994516,11cyclictest0-21swapper/008:03:070
320994431,7cyclictest19844-21aten2.4-expect09:57:413
320994430,8cyclictest25898-21cpuspeed11:17:403
320994430,8cyclictest21314-21cut08:52:453
32099443,36cyclictest0-21swapper/309:37:513
32099443,36cyclictest0-21swapper/308:22:433
32099443,27cyclictest0-21swapper/307:22:463
320994427,10cyclictest1727-21aten_r1power_en10:27:403
318994412,11cyclictest114550irq/46-eth007:02:341
31299443,37cyclictest0-21swapper/008:34:200
282202442,36sleep00-21swapper/010:12:560
320994328,9cyclictest16557-21unixbench-2d12:02:573
320994322,10cyclictest0-21swapper/309:51:253
31999435,14cyclictest0-21swapper/212:22:372
312994323,13cyclictest0-21swapper/007:33:000
32099423,25cyclictest0-21swapper/311:02:363
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional