You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-30 - 23:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Tue Dec 30, 2025 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7101992724,2cyclictest23203-21dump-pmu-power08:50:010
710499250,1cyclictest14109-21awk10:45:012
7105992420,3cyclictest0-21swapper/310:55:423
710499240,22cyclictest313-21systemd-journal08:00:012
710499240,1cyclictest0-21swapper/211:00:462
710199240,23cyclictest26184-21munin-plugin-st11:10:000
7105992320,2cyclictest313-21systemd-journal08:35:003
710599230,22cyclictest29704-21awk10:10:003
710499231,21cyclictest5230-21apt09:20:002
710499230,22cyclictest27330-21munin-plugin-st10:05:012
710199230,22cyclictest24341-21sh07:45:000
7105992220,1cyclictest9447-21munin-run07:15:013
7105992220,1cyclictest24903-21latency_hist10:00:013
710599220,7cyclictest0-21swapper/312:30:443
710599220,21cyclictest14431-21apt-get10:45:443
710599220,21cyclictest0-21swapper/308:05:433
710599220,21cyclictest0-21swapper/308:05:433
710599220,21cyclictest0-21swapper/307:35:423
7104992220,1cyclictest13593-21awk08:30:002
7104992220,1cyclictest0-21swapper/212:20:012
7104992220,1cyclictest0-21swapper/211:25:012
710499220,21cyclictest0-21swapper/209:00:432
7103992220,1cyclictest313-21systemd-journal10:10:001
7103992220,1cyclictest313-21systemd-journal10:00:011
7103992220,1cyclictest0-21swapper/110:25:001
7103992217,4cyclictest19223-21apt-get10:55:421
710399220,21cyclictest28380-21apt09:00:431
710399220,21cyclictest20821-21munin-run08:45:001
710399220,21cyclictest15990-21cron08:35:001
710399220,21cyclictest12769-21apt-get09:35:441
710399220,1cyclictest364-21cron09:10:011
710399220,0cyclictest0-21swapper/108:20:451
710399220,0cyclictest0-21swapper/107:20:421
7101992220,1cyclictest22442-21latency_hist09:55:010
7101992220,1cyclictest10578-21perf11:45:000
710199220,21cyclictest10069-21munin-run09:30:000
710199220,21cyclictest0-21swapper/010:10:450
710199220,21cyclictest0-21swapper/009:10:440
7105992121,0cyclictest0-21swapper/308:45:433
7105992120,1cyclictest18723-21apt-get08:40:413
7105992120,1cyclictest0-21swapper/308:55:433
7105992119,1cyclictest30467-21turbostat09:05:013
7105992119,1cyclictest0-21swapper/307:50:463
7105992116,5cyclictest0-21swapper/310:35:443
710599210,21cyclictest0-21swapper/308:35:443
710599210,20cyclictest16956-21apt-get07:30:433
710599210,1cyclictest21643-21apt-get11:00:453
710599210,0cyclictest0-21swapper/309:05:443
710599210,0cyclictest0-21swapper/307:25:423
710599210,0cyclictest0-21swapper/307:20:423
7104992120,1cyclictest0-21swapper/209:25:432
7104992119,1cyclictest6400-21munin-run08:15:012
7104992119,1cyclictest4824-21apt-get10:25:452
7104992119,1cyclictest30990-21perf11:20:012
7104992119,1cyclictest29201-21perf07:55:002
7104992119,1cyclictest20856-21/usr/sbin/munin12:05:452
7104992119,1cyclictest0-21swapper/211:40:442
7104992119,1cyclictest0-21swapper/207:50:002
7104992118,2cyclictest9307-21dump-pmu-power10:35:012
710499210,20cyclictest736-21apt-get09:10:432
710499210,20cyclictest22264-21apt-get07:40:422
710499210,20cyclictest20073-21grep09:50:012
710499210,20cyclictest12163-21apt-get07:20:422
710499210,20cyclictest0-21swapper/207:15:442
710499210,20cyclictest0-21swapper/207:15:442
710499210,1cyclictest0-21swapper/210:15:002
710499210,0cyclictest0-21swapper/211:45:442
7103992121,0cyclictest25234-21apt-get10:00:421
7103992121,0cyclictest0-21swapper/110:35:441
7103992120,0cyclictest11202-21apt-get11:45:021
7103992119,1cyclictest22964-21latency_hist12:10:011
710399210,21cyclictest0-21swapper/111:00:461
710399210,21cyclictest0-21swapper/109:40:021
710399210,21cyclictest0-21swapper/107:35:021
710399210,20cyclictest7351-21apt-get07:10:421
710399210,20cyclictest3676-21apt-get11:30:431
710399210,20cyclictest0-21swapper/111:40:431
710399210,20cyclictest0-21swapper/110:35:011
710399210,20cyclictest0-21swapper/109:25:441
710399210,0cyclictest6693-21apt-get08:15:421
710399210,0cyclictest30646-21munin-run12:25:011
710399210,0cyclictest0-21swapper/112:25:431
7101992121,0cyclictest16308-21apt-get08:35:440
7101992120,1cyclictest16828-21apt-get10:50:430
7101992119,1cyclictest6393-21apt08:15:010
710199210,21cyclictest0-21swapper/011:20:420
710199210,21cyclictest0-21swapper/009:35:440
710199210,20cyclictest0-21swapper/011:45:440
710199210,20cyclictest0-21swapper/007:30:430
710199210,0cyclictest15493-21apt-get09:40:020
710199210,0cyclictest0-21swapper/012:05:450
710199210,0cyclictest0-21swapper/009:55:440
710199210,0cyclictest0-21swapper/009:45:430
7105992020,0cyclictest6075-21apt-get11:35:443
7105992020,0cyclictest0-21swapper/312:25:433
7105992020,0cyclictest0-21swapper/311:45:443
7105992020,0cyclictest0-21swapper/311:20:423
7105992020,0cyclictest0-21swapper/309:40:433
7105992020,0cyclictest0-21swapper/309:35:443
7105992020,0cyclictest0-21swapper/309:15:443
7105992019,1cyclictest0-21swapper/308:20:453
7105992019,0cyclictest0-21swapper/311:25:443
710599200,20cyclictest0-21swapper/311:15:443
710599200,19cyclictest0-21swapper/312:15:433
710599200,0cyclictest0-21swapper/310:30:443
7104992020,0cyclictest0-21swapper/211:25:442
7104992020,0cyclictest0-21swapper/210:15:432
7104992020,0cyclictest0-21swapper/208:05:422
7104992020,0cyclictest0-21swapper/208:05:422
7104992019,1cyclictest0-21swapper/207:25:422
7104992019,0cyclictest2426-21apt-get10:20:432
7104992018,1cyclictest0-21swapper/208:50:012
7104992017,2cyclictest29686-21cron10:10:002
710499200,0cyclictest0-21swapper/212:20:432
710499200,0cyclictest0-21swapper/211:05:442
710499200,0cyclictest0-21swapper/210:50:432
710499200,0cyclictest0-21swapper/210:45:442
7103992020,0cyclictest0-21swapper/112:35:021
7103992020,0cyclictest0-21swapper/112:10:441
7103992020,0cyclictest0-21swapper/111:25:431
7103992020,0cyclictest0-21swapper/108:05:431
7103992020,0cyclictest0-21swapper/108:05:431
7103992020,0cyclictest0-21swapper/107:30:431
7103992019,1cyclictest32121-21munin-run10:15:001
710399200,20cyclictest0-21swapper/109:20:431
710399200,20cyclictest0-21swapper/107:25:431
7101992020,0cyclictest0-21swapper/012:30:450
7101992020,0cyclictest0-21swapper/008:20:450
7101992020,0cyclictest0-21swapper/008:05:430
7101992020,0cyclictest0-21swapper/008:05:430
7101992020,0cyclictest0-21swapper/007:45:430
7101992020,0cyclictest0-21swapper/007:10:420
7101992018,1cyclictest28381-21apt-get09:00:430
710199200,20cyclictest4517-21perf10:25:000
710199200,0cyclictest0-21swapper/010:30:440
710199200,0cyclictest0-21swapper/009:05:440
7105991919,0cyclictest20889-21apt-get12:05:453
7105991919,0cyclictest0-21swapper/312:10:443
7105991919,0cyclictest0-21swapper/311:50:453
7105991919,0cyclictest0-21swapper/310:20:433
7105991919,0cyclictest0-21swapper/307:40:423
7105991918,1cyclictest32418-21apt-get10:15:443
7105991918,0cyclictest0-21swapper/311:40:443
710599190,1cyclictest6371-21sh08:15:013
7104991919,0cyclictest0-21swapper/212:00:442
7104991919,0cyclictest0-21swapper/211:35:432
7104991919,0cyclictest0-21swapper/209:35:442
7104991917,1cyclictest551-21idleruntime-cro12:30:012
7103991919,0cyclictest27115-21apt-get07:50:461
7103991919,0cyclictest23514-21apt-get08:50:431
7103991919,0cyclictest0-21swapper/110:45:441
7103991919,0cyclictest0-21swapper/108:35:441
7103991918,1cyclictest28878-21apt-get11:15:441
7103991918,1cyclictest21117-21apt-get08:45:431
7103991917,1cyclictest25964-21apt-get08:55:431
710399190,8cyclictest0-21swapper/112:30:441
7101991919,0cyclictest23680-21apt-get12:10:440
7101991919,0cyclictest0-21swapper/012:35:470
7101991919,0cyclictest0-21swapper/012:00:430
7101991919,0cyclictest0-21swapper/011:35:440
7101991919,0cyclictest0-21swapper/011:10:430
7101991919,0cyclictest0-21swapper/010:55:420
7101991919,0cyclictest0-21swapper/010:40:440
7101991919,0cyclictest0-21swapper/010:15:440
7101991919,0cyclictest0-21swapper/009:20:430
7101991919,0cyclictest0-21swapper/008:55:430
7101991919,0cyclictest0-21swapper/008:30:440
7101991919,0cyclictest0-21swapper/008:15:420
7101991919,0cyclictest0-21swapper/007:35:030
7101991918,1cyclictest26129-21apt-get12:15:430
7101991918,1cyclictest0-21swapper/011:30:430
7101991918,1cyclictest0-21swapper/010:00:420
7101991918,1cyclictest0-21swapper/008:50:430
710199190,0cyclictest3157-21apt-get09:15:440
710199190,0cyclictest27631-21apt-get10:05:430
7105991818,0cyclictest0-21swapper/310:50:443
7105991818,0cyclictest0-21swapper/309:30:423
7105991818,0cyclictest0-21swapper/308:50:433
7105991818,0cyclictest0-21swapper/308:15:423
7105991817,1cyclictest30616-21awk12:25:013
7105991817,0cyclictest0-21swapper/309:10:433
7105991816,1cyclictest20589-21perf12:05:013
7105991815,3cyclictest26483-21apt-get11:10:433
7104991818,0cyclictest0-21swapper/208:00:432
7104991817,0cyclictest20372-21apt-get09:50:452
7104991816,1cyclictest9620-21apt-get10:35:442
7104991816,1cyclictest17677-21munin-run09:45:012
7103991818,0cyclictest0-21swapper/111:20:421
7103991818,0cyclictest0-21swapper/111:10:431
7103991818,0cyclictest0-21swapper/109:15:441
7103991818,0cyclictest0-21swapper/109:10:441
7103991817,1cyclictest13686-21apt-get11:50:451
7103991817,1cyclictest0-21swapper/111:35:441
7101991818,0cyclictest0-21swapper/012:20:430
7101991818,0cyclictest0-21swapper/011:00:460
7101991818,0cyclictest0-21swapper/010:25:450
7101991818,0cyclictest0-21swapper/009:30:420
7101991818,0cyclictest0-21swapper/008:00:440
710199180,0cyclictest0-21swapper/011:50:450
7105991717,0cyclictest0-21swapper/312:35:023
7105991717,0cyclictest0-21swapper/309:25:433
7105991717,0cyclictest0-21swapper/307:55:433
7105991716,1cyclictest0-21swapper/310:00:413
7105991716,0cyclictest0-21swapper/310:10:453
7105991715,1cyclictest0-21swapper/309:50:453
7104991717,0cyclictest23680-21apt-get12:10:442
7104991717,0cyclictest0-21swapper/211:10:432
7104991717,0cyclictest0-21swapper/210:55:432
7104991717,0cyclictest0-21swapper/209:05:442
7104991717,0cyclictest0-21swapper/208:30:442
7104991717,0cyclictest0-21swapper/207:30:432
710399176,10cyclictest6372-21idleruntime-cro08:15:011
7103991717,0cyclictest18494-21apt-get12:00:441
7103991717,0cyclictest0-21swapper/112:15:431
7103991717,0cyclictest0-21swapper/110:25:451
7103991714,3cyclictest0-21swapper/111:55:451
710399170,1cyclictest12457-21cat09:35:001
710399170,16cyclictest11431-21sshd08:25:031
710399170,16cyclictest0-21swapper/107:45:461
7101991717,0cyclictest0-21swapper/012:25:430
7101991717,0cyclictest0-21swapper/011:55:440
7101991717,0cyclictest0-21swapper/007:50:460
7105991616,0cyclictest0-21swapper/307:15:433
7105991616,0cyclictest0-21swapper/307:15:433
7104991616,0cyclictest9123-21apt-get08:20:452
7104991614,1cyclictest0-21swapper/208:55:012
710499160,7cyclictest0-21swapper/209:30:422
7103991616,0cyclictest0-21swapper/107:40:421
7103991615,1cyclictest0-21swapper/111:05:431
7103991615,0cyclictest0-21swapper/110:50:441
7103991614,1cyclictest0-21swapper/108:00:011
710399160,0cyclictest0-21swapper/107:15:231
710399160,0cyclictest0-21swapper/107:15:231
7101991616,0cyclictest0-21swapper/007:25:420
7101991614,1cyclictest12160-21hddtemp_smartct08:25:460
7105991515,0cyclictest0-21swapper/310:25:453
7105991514,1cyclictest0-21swapper/311:34:023
7105991513,1cyclictest0-21swapper/307:50:003
7104991515,0cyclictest0-21swapper/209:20:432
7104991514,1cyclictest19744-21apt-get07:35:432
7104991513,1cyclictest916-21apt-get12:30:442
7104991513,1cyclictest0-21swapper/212:35:022
7103991514,1cyclictest0-21swapper/108:01:291
7103991512,3cyclictest0-21swapper/110:15:441
710399150,15cyclictest0-21swapper/109:46:101
7101991513,1cyclictest0-21swapper/010:45:460
710199150,1cyclictest30302-21awk07:55:460
7105991411,3cyclictest0-21swapper/311:55:443
710599140,14cyclictest0-21swapper/310:41:033
710599140,14cyclictest0-21swapper/309:20:053
710599140,14cyclictest0-21swapper/308:25:363
710599140,0cyclictest0-21swapper/309:45:033
710599140,0cyclictest0-21swapper/308:01:473
7104991414,0cyclictest0-21swapper/208:35:222
710499140,14cyclictest0-21swapper/211:55:102
710499140,14cyclictest0-21swapper/211:50:162
710499140,14cyclictest0-21swapper/211:31:122
710499140,14cyclictest0-21swapper/209:55:232
710499140,14cyclictest0-21swapper/208:55:212
710499140,14cyclictest0-21swapper/208:40:282
710499140,14cyclictest0-21swapper/208:16:092
710499140,14cyclictest0-21swapper/207:10:512
7103991414,0cyclictest12039-21apt-get10:40:441
7103991413,1cyclictest0-21swapper/109:50:451
7101991414,0cyclictest0-21swapper/011:26:490
7101991414,0cyclictest0-21swapper/010:35:050
7101991414,0cyclictest0-21swapper/007:20:060
710199140,14cyclictest0-21swapper/011:16:120
710199140,14cyclictest0-21swapper/008:40:120
710199140,14cyclictest0-21swapper/007:16:390
710199140,14cyclictest0-21swapper/007:16:390
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional