You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-15 - 18:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Sat Nov 15, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1375699903,80cyclictest0-21swapper/109:15:191
285072890,2sleep041ktimersoftd/012:05:290
1375699874,62cyclictest0-21swapper/109:48:421
1375699865,75cyclictest0-21swapper/107:50:211
1375699864,62cyclictest20465-21chrt08:25:381
1375699855,60cyclictest0-21swapper/111:55:331
1375699845,51cyclictest6165-21taskset11:17:151
1375699844,59cyclictest0-21swapper/107:30:441
1375699834,72cyclictest0-21swapper/111:10:141
1375699825,71cyclictest0-21swapper/109:05:191
1375699825,58cyclictest0-21swapper/107:23:191
1375699815,45cyclictest0-21swapper/112:10:421
1375699815,45cyclictest0-21swapper/112:10:421
1375699814,70cyclictest0-21swapper/109:55:171
1375699814,42cyclictest0-21swapper/111:50:401
1375699813,9cyclictest0-21swapper/108:25:201
13752998132,42cyclictest14423-21grep10:25:500
1375699805,68cyclictest0-21swapper/108:40:391
1375699805,68cyclictest0-21swapper/108:40:391
1375699805,55cyclictest0-21swapper/109:41:051
1375699804,69cyclictest0-21swapper/112:30:341
1375699804,10cyclictest0-21swapper/109:57:191
1375699803,8cyclictest12378-21if_eth011:30:371
1375299805,68cyclictest4409-21gzip12:20:410
13752998032,42cyclictest1194-21sed07:45:390
1375699794,56cyclictest0-21swapper/107:45:221
13756997938,32cyclictest27275-21tr10:55:321
13756997927,43cyclictest24257-21fschecks_time07:25:421
1375299794,69cyclictest0-21swapper/010:35:150
13752997922,39cyclictest0-21swapper/007:15:230
13752997922,39cyclictest0-21swapper/007:15:230
1375699785,66cyclictest0-21swapper/107:40:231
1375699785,65cyclictest32133-21cpuspeed_turbos08:50:351
1375699784,7cyclictest1369-21runrttasks10:49:581
1375699784,7cyclictest1369-21runrttasks10:49:581
1375699784,67cyclictest0-21swapper/108:15:481
1375699784,66cyclictest0-21swapper/111:00:481
13752997833,39cyclictest6585-21uname07:55:440
13752997832,40cyclictest32089-21tr11:05:300
1375699774,66cyclictest0-21swapper/108:40:211
1375699774,65cyclictest0-21swapper/107:55:221
13756997730,41cyclictest13490-21gltestperf10:25:361
1375699773,9cyclictest20867-21sshd07:20:101
1375699773,7cyclictest4025-21iostat_ios12:20:341
1375699773,56cyclictest13746-21cat09:20:191
1375699773,10cyclictest0-21swapper/112:05:341
1375299774,66cyclictest0-21swapper/008:35:400
1375699768,46cyclictest114550irq/46-eth011:35:431
1375699765,65cyclictest0-21swapper/110:10:291
1375699765,53cyclictest0-21swapper/110:40:501
1375699763,12cyclictest0-21swapper/109:25:311
1375699763,12cyclictest0-21swapper/109:25:311
13752997620,50cyclictest17720-21python09:25:500
13752997620,50cyclictest17720-21python09:25:500
1375699754,64cyclictest0-21swapper/108:45:491
1375699754,63cyclictest0-21swapper/110:50:491
13756997527,41cyclictest27297-21timerandwakeup12:00:431
13756997527,41cyclictest11073-21gltestperf10:20:371
13756997518,35cyclictest0-21swapper/111:25:301
13756997517,48cyclictest1215-21fschecks_time12:15:301
13756997516,53cyclictest8352-21timerwakeupswit09:05:511
1375299755,63cyclictest39101sshd11:14:010
1375299754,64cyclictest0-21swapper/009:40:430
13752997536,32cyclictest21284-21cpuspeed_turbos11:50:270
13752997531,38cyclictest13544-21gltestperf08:10:410
13752997530,39cyclictest2306-21unixbench_multi12:15:450
13752997527,43cyclictest25989-21aten_r1power_fr07:30:350
1375699745,59cyclictest1369-21runrttasks10:05:501
1375699745,59cyclictest1369-21runrttasks10:05:501
1375699745,47cyclictest0-21swapper/111:10:261
1375699744,63cyclictest0-21swapper/110:00:381
1375699743,8cyclictest14046-21tail09:20:321
13756997426,42cyclictest14128-21aten_r1power_vo07:05:361
13756997417,41cyclictest0-21swapper/107:05:241
1375299745,63cyclictest0-21swapper/012:05:120
1375299744,64cyclictest0-21swapper/011:35:140
13752997431,38cyclictest12302-21gltestperf09:15:390
1375699739,8cyclictest114550irq/46-eth010:15:311
1375699736,59cyclictest9656-21unixbench_multi08:00:551
1375699735,57cyclictest16202-21aten_r1power_cu11:40:251
1375699734,62cyclictest0-21swapper/107:11:511
1375699734,62cyclictest0-21swapper/107:11:511
1375699734,10cyclictest0-21swapper/108:00:001
1375699733,8cyclictest20505-21chrt11:47:321
1375699733,7cyclictest7599-21cat12:30:111
1375299734,63cyclictest0-21swapper/012:10:250
1375299734,63cyclictest0-21swapper/012:10:250
1375299734,62cyclictest1369-21runrttasks07:20:570
1375299734,61cyclictest4637-21ls07:50:540
1375299734,61cyclictest0-21swapper/010:45:150
13752997330,38cyclictest28538-21cpuspeed_turbos09:50:330
1375299733,63cyclictest0-21swapper/008:20:210
1375699725,60cyclictest19140-21unixbench_multi10:35:521
1375699724,59cyclictest2769-21gltestperf08:55:401
1375699723,9cyclictest0-21swapper/108:10:381
1375699723,7cyclictest16255-21needreboot10:30:431
13756997225,40cyclictest7347-21fschecks_time11:20:341
13756997220,46cyclictest22330-21aten_r1power_en08:30:331
1375299725,61cyclictest0-21swapper/007:05:360
1375299724,62cyclictest0-21swapper/009:40:190
1375299724,61cyclictest3890-21chrt08:57:450
1375299724,61cyclictest0-21swapper/011:55:340
1375299724,58cyclictest1369-21runrttasks11:17:150
13752997226,41cyclictest17622-21aten_r1power_po08:20:330
1375699713,62cyclictest21358-21tr09:35:351
1375299716,36cyclictest24600-21aten_r1power_cu10:50:270
1375299715,58cyclictest0-21swapper/009:05:520
1375299713,61cyclictest567-21needreboot08:50:450
1375299713,52cyclictest0-21swapper/010:13:450
13752997114,40cyclictest0-21swapper/010:20:160
1375699704,60cyclictest0-21swapper/108:05:531
1375699703,10cyclictest0-21swapper/109:30:381
1375299704,60cyclictest0-21swapper/011:41:290
1375299703,60cyclictest0-21swapper/007:25:370
1375299703,58cyclictest0-21swapper/009:56:180
13752997027,37cyclictest19345-21cut07:15:430
13752997025,39cyclictest18740-21cpu11:45:290
1375299696,56cyclictest31152-21uptime11:00:490
1375299693,60cyclictest9362-21gzip08:00:510
13752996926,38cyclictest0-21swapper/012:35:110
13752996922,41cyclictest9185-21cpu11:25:320
1375299684,30cyclictest0-21swapper/010:20:390
1375299684,30cyclictest0-21swapper/007:40:400
1375299683,58cyclictest6949-21cpuspeed_turbos11:20:290
13752996825,38cyclictest1369-21runrttasks10:08:350
13752996825,38cyclictest1369-21runrttasks10:08:350
13752996823,39cyclictest1275-21fschecks_time10:00:360
13752996820,42cyclictest22420-21basename08:30:340
97712670,4sleep29890-21head11:25:382
243799670,9rtkit-daemon0-21swapper/208:25:562
1375299676,53cyclictest0-21swapper/010:45:400
1375299676,53cyclictest0-21swapper/010:45:400
1375299675,57cyclictest0-21swapper/012:25:350
1375299675,56cyclictest0-21swapper/009:50:030
1375299675,55cyclictest0-21swapper/010:57:010
1375299674,56cyclictest0-21swapper/010:35:300
13752996722,39cyclictest15379-21switchtime09:20:500
13752996623,38cyclictest19916-21sh08:25:330
13752996621,39cyclictest1369-21runrttasks08:44:200
13752996621,39cyclictest1369-21runrttasks08:44:200
1375299654,56cyclictest0-21swapper/011:35:360
1375299654,54cyclictest0-21swapper/009:30:510
1375299653,43cyclictest111652gltestperf08:05:410
1375299645,54cyclictest0-21swapper/009:00:320
1375299643,55cyclictest29592-21telnet08:45:330
1375299635,35cyclictest0-21swapper/007:37:590
243799620,5rtkit-daemon2436-21rtkit-daemon08:43:183
243799620,5rtkit-daemon2436-21rtkit-daemon08:43:183
13752996223,34cyclictest9402-21cpuspeed_turbos09:10:360
243799610,4rtkit-daemon2436-21rtkit-daemon12:04:253
167892617,45sleep20-21swapper/207:10:402
167892617,45sleep20-21swapper/207:10:402
13758996143,12cyclictest23054-21gltestperf10:45:373
13758996143,12cyclictest23054-21gltestperf10:45:373
243799600,4rtkit-daemon2436-21rtkit-daemon11:31:273
243799600,4rtkit-daemon2436-21rtkit-daemon09:17:393
243799600,4rtkit-daemon2436-21rtkit-daemon07:46:503
1375899595,23cyclictest16514-21telnet07:10:363
1375899595,23cyclictest16514-21telnet07:10:363
243799580,6rtkit-daemon2436-21rtkit-daemon10:59:132
243799580,4rtkit-daemon2436-21rtkit-daemon08:34:263
1375899588,26cyclictest0-21swapper/308:15:343
1375299585,45cyclictest13753-21idleruntime-cro07:05:230
89772573,12sleep20-21swapper/208:00:452
243799570,5rtkit-daemon2436-21rtkit-daemon09:30:462
243799570,4rtkit-daemon2436-21rtkit-daemon11:54:483
1375899574,30cyclictest0-21swapper/310:20:163
243799560,5rtkit-daemon2436-21rtkit-daemon11:05:432
243799560,4rtkit-daemon2436-21rtkit-daemon11:27:123
13758995639,10cyclictest24723-21cut10:50:293
13758995622,14cyclictest0-21swapper/312:30:473
243799550,5rtkit-daemon2436-21rtkit-daemon07:50:353
243799550,5rtkit-daemon2436-21rtkit-daemon07:25:442
243799550,5rtkit-daemon2436-21rtkit-daemon07:24:142
1375899555,30cyclictest0-21swapper/308:00:363
13758995515,22cyclictest0-21swapper/309:55:313
243799540,5rtkit-daemon2436-21rtkit-daemon11:00:352
243799540,5rtkit-daemon2436-21rtkit-daemon10:40:363
243799540,5rtkit-daemon2436-21rtkit-daemon09:10:553
23942542,10sleep20-21swapper/212:15:512
190612543,43sleep30-21swapper/308:20:533
172952544,43sleep30-21swapper/309:25:423
172952544,43sleep30-21swapper/309:25:423
1375899544,22cyclictest0-21swapper/311:47:003
1375899543,36cyclictest5490-21munin-node10:10:293
13758995420,14cyclictest0-21swapper/307:15:463
13758995417,13cyclictest0-21swapper/307:08:163
13758995416,14cyclictest0-21swapper/309:20:453
52572532,10sleep20-21swapper/207:54:452
243799530,6rtkit-daemon2436-21rtkit-daemon12:20:472
243799530,5rtkit-daemon2436-21rtkit-daemon12:11:182
243799530,5rtkit-daemon2436-21rtkit-daemon12:11:182
243799530,5rtkit-daemon2436-21rtkit-daemon10:50:362
243799530,5rtkit-daemon2436-21rtkit-daemon10:20:373
243799530,5rtkit-daemon2436-21rtkit-daemon09:00:403
243799530,5rtkit-daemon2436-21rtkit-daemon08:10:572
243799530,5rtkit-daemon2436-21rtkit-daemon07:08:592
179162533,10sleep218034-21df08:20:362
1375899534,40cyclictest0-21swapper/308:45:313
1375899534,29cyclictest0-21swapper/311:20:423
1375899534,29cyclictest0-21swapper/311:10:143
13758995321,14cyclictest0-21swapper/310:55:443
13758995321,13cyclictest0-21swapper/309:43:413
13758995321,13cyclictest0-21swapper/307:30:593
13758995320,13cyclictest0-21swapper/308:25:443
13758995319,15cyclictest0-21swapper/307:40:483
13758995319,14cyclictest0-21swapper/309:45:483
13758995317,12cyclictest0-21swapper/312:14:063
13758995317,12cyclictest0-21swapper/312:14:063
13758995315,14cyclictest0-21swapper/308:05:393
243799520,6rtkit-daemon2436-21rtkit-daemon11:18:572
243799520,5rtkit-daemon2436-21rtkit-daemon12:04:552
243799520,5rtkit-daemon2436-21rtkit-daemon09:21:542
243799520,5rtkit-daemon2436-21rtkit-daemon07:30:362
243799520,4rtkit-daemon2436-21rtkit-daemon09:05:473
206572522,42sleep20-21swapper/207:17:252
1375899525,29cyclictest0-21swapper/308:55:493
1375899525,26cyclictest0-21swapper/310:36:033
1375899524,29cyclictest0-21swapper/312:15:403
1375899524,29cyclictest0-21swapper/307:55:503
1375899524,29cyclictest0-21swapper/307:35:503
1375899524,25cyclictest0-21swapper/310:08:443
1375899524,25cyclictest0-21swapper/310:08:443
13758995234,13cyclictest32440-21fschecks_count08:50:383
13758995233,13cyclictest24181-21cut11:55:333
1375899523,29cyclictest0-21swapper/311:10:393
13758995226,19cyclictest15870-21gltestperf10:30:373
1375899522,34cyclictest0-21swapper/307:20:503
13758995219,14cyclictest0-21swapper/311:15:263
13758995218,13cyclictest0-21swapper/310:25:373
13758995218,13cyclictest0-21swapper/307:26:003
13758995217,13cyclictest0-21swapper/310:00:423
13757995230,9cyclictest25415-21yum-updatesd-he11:59:082
180222514,38sleep20-21swapper/210:35:372
1375899515,27cyclictest1369-21runrttasks12:20:233
1375899514,29cyclictest0-21swapper/311:35:313
1375899514,26cyclictest0-21swapper/312:05:463
1375899514,26cyclictest0-21swapper/311:00:293
1375899514,26cyclictest0-21swapper/307:05:243
13758995117,14cyclictest0-21swapper/309:50:383
13758995116,12cyclictest0-21swapper/311:45:143
1375799514,38cyclictest0-21swapper/209:55:162
13757995133,11cyclictest7265-21unixbench_multi07:55:542
1375799513,42cyclictest0-21swapper/208:40:402
1375799513,42cyclictest0-21swapper/208:40:402
243799500,4rtkit-daemon2436-21rtkit-daemon10:45:512
243799500,4rtkit-daemon2436-21rtkit-daemon10:45:512
243799500,4rtkit-daemon2436-21rtkit-daemon09:28:392
243799500,4rtkit-daemon2436-21rtkit-daemon09:28:392
1375899505,27cyclictest0-21swapper/309:38:413
1375899505,26cyclictest0-21swapper/312:29:083
13758995033,12cyclictest24662-21munin-node08:35:423
1375799503,28cyclictest0-21swapper/209:05:452
243799490,5rtkit-daemon2436-21rtkit-daemon12:05:332
243799490,5rtkit-daemon2436-21rtkit-daemon10:43:512
13757994933,10cyclictest32381-21unixbench_multi07:40:572
13757994932,10cyclictest30958-21cpuspeed_turbos09:55:342
1375799493,28cyclictest0-21swapper/210:25:352
75192482,9sleep20-21swapper/210:13:492
243799480,4rtkit-daemon2436-21rtkit-daemon07:35:512
229182482,10sleep20-21swapper/209:36:452
1375899484,26cyclictest0-21swapper/308:13:273
1375799484,27cyclictest0-21swapper/211:45:322
1375799482,39cyclictest0-21swapper/208:55:322
85672472,11sleep20-21swapper/211:22:322
49522472,9sleep20-21swapper/210:06:232
49522472,9sleep20-21swapper/210:06:232
169742472,10sleep20-21swapper/210:32:352
13758994727,13cyclictest0-21swapper/309:30:363
13757994728,13cyclictest0-21swapper/210:20:422
26072462,10sleep20-21swapper/207:46:142
159112462,9sleep20-21swapper/211:39:532
13757994632,8cyclictest6266-21gltestperf12:25:332
1375799463,27cyclictest0-21swapper/211:10:342
1375799463,27cyclictest0-21swapper/208:15:352
1375799463,26cyclictest0-21swapper/211:50:292
1375799463,26cyclictest0-21swapper/209:15:312
1375799463,19cyclictest0-21swapper/208:32:482
97662453,35sleep20-21swapper/210:16:592
5262452,10sleep20-21swapper/210:00:192
13757994531,9cyclictest16547-21tr11:40:282
1375799453,26cyclictest0-21swapper/211:30:312
1375799453,26cyclictest0-21swapper/209:03:332
277882442,36sleep20-21swapper/209:48:022
13757994419,10cyclictest29459-21telnet08:45:312
1375799434,12cyclictest0-21swapper/212:30:432
13757994326,12cyclictest0-21swapper/208:05:552
13757994319,9cyclictest9237-21telnet09:10:342
13757994311,20cyclictest1243-21snmpd08:53:332
13757994221,8cyclictest24662-21munin-node08:35:402
13757994216,12cyclictest0-21swapper/209:40:302
1375799382,23cyclictest0-21swapper/207:05:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional