You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-28 - 18:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Tue Apr 28, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1749992523,1cyclictest28151-21sendmail00:20:013
174999250,24cyclictest0-21swapper/300:10:473
1741992525,0cyclictest0-21swapper/123:28:351
173799250,25cyclictest0-21swapper/023:21:360
173799250,24cyclictest13259-21cat23:00:530
173799250,1cyclictest27424-21diskmemload00:05:180
174999240,2cyclictest14611-21awk00:05:503
174999240,23cyclictest8106-21ssh21:53:533
174999240,23cyclictest0-21swapper/322:39:473
1745992422,1cyclictest0-21swapper/223:18:322
1745992422,1cyclictest0-21swapper/223:06:062
174599240,23cyclictest0-21swapper/223:37:282
174599240,23cyclictest0-21swapper/222:57:292
174599240,23cyclictest0-21swapper/222:30:242
174599240,23cyclictest0-21swapper/221:59:412
174599240,23cyclictest0-21swapper/221:35:442
174599240,23cyclictest0-21swapper/221:26:312
174599240,23cyclictest0-21swapper/200:15:302
174599240,23cyclictest0-21swapper/200:10:462
174599240,0cyclictest27424-21diskmemload22:36:212
1741992423,1cyclictest27424-21diskmemload22:52:151
1741992422,1cyclictest0-21swapper/100:34:021
1741992422,1cyclictest0-21swapper/100:23:391
1741992421,2cyclictest16660-21turbostat.cron00:40:001
174199240,23cyclictest3180-21apt-get23:55:451
174199240,23cyclictest18655-21apt-get00:10:461
1737992423,1cyclictest12401-21ssh23:31:490
173799240,24cyclictest0-21swapper/000:22:370
173799240,0cyclictest0-21swapper/023:43:540
1749992323,0cyclictest0-21swapper/323:56:263
1749992323,0cyclictest0-21swapper/323:31:343
1749992323,0cyclictest0-21swapper/300:31:443
1749992323,0cyclictest0-21swapper/300:23:013
1749992322,1cyclictest10543-21rm22:59:173
1749992321,1cyclictest20685-21munin-run21:35:003
1749992321,1cyclictest0-21swapper/321:55:343
1749992321,1cyclictest0-21swapper/321:35:533
174999230,23cyclictest27424-21diskmemload23:52:033
174999230,23cyclictest27424-21diskmemload21:45:543
174999230,23cyclictest27424-21diskmemload21:45:543
174999230,23cyclictest0-21swapper/323:25:073
174999230,23cyclictest0-21swapper/322:24:213
174999230,23cyclictest0-21swapper/322:18:433
174999230,23cyclictest0-21swapper/322:07:523
174999230,23cyclictest0-21swapper/322:02:113
174999230,22cyclictest31452-21munin-run23:20:013
174999230,22cyclictest24570-21ssh23:44:533
174999230,22cyclictest15039-21ssh22:31:173
174999230,22cyclictest14705-21rm23:02:443
174999230,22cyclictest0-21swapper/321:22:463
174999230,0cyclictest0-21swapper/323:36:083
174999230,0cyclictest0-21swapper/322:47:513
174999230,0cyclictest0-21swapper/321:29:233
174999230,0cyclictest0-21swapper/321:10:303
1745992323,0cyclictest0-21swapper/222:40:312
1745992323,0cyclictest0-21swapper/221:44:542
1745992323,0cyclictest0-21swapper/221:21:502
1745992322,1cyclictest27424-21diskmemload22:00:072
1745992322,1cyclictest0-21swapper/223:49:222
1745992321,1cyclictest3003-21perf20:20:012
174599230,23cyclictest0-21swapper/222:25:502
174599230,22cyclictest7815-21munin-run00:00:012
174599230,22cyclictest15440-21awk20:45:002
174599230,22cyclictest0-21swapper/223:11:362
174599230,22cyclictest0-21swapper/200:21:042
174599230,22cyclictest0-21swapper/200:05:222
174599230,0cyclictest0-21swapper/223:31:042
174599230,0cyclictest0-21swapper/223:02:542
174599230,0cyclictest0-21swapper/222:08:272
174599230,0cyclictest0-21swapper/200:33:232
1741992323,0cyclictest0-21swapper/122:15:321
1741992323,0cyclictest0-21swapper/122:07:561
1741992323,0cyclictest0-21swapper/121:35:291
174199232,1cyclictest14994-1kworker/u9:121:15:471
1741992321,1cyclictest26660-21ssh23:15:121
1741992321,1cyclictest14925-21ssh23:35:281
1741992321,1cyclictest0-21swapper/123:10:401
1741992321,1cyclictest0-21swapper/122:45:341
1741992321,1cyclictest0-21swapper/121:24:191
1741992320,2cyclictest5561-21ssh22:22:251
174199230,2cyclictest2009-21sh23:21:331
174199230,23cyclictest0-21swapper/123:34:351
174199230,22cyclictest6627-21apt-get22:55:441
174199230,22cyclictest27424-21diskmemload23:50:521
174199230,22cyclictest14219-21ssh21:27:451
174199230,22cyclictest10617-21ntpq21:55:501
174199230,22cyclictest0-21swapper/100:15:511
174199230,21cyclictest0-21swapper/122:35:301
174199230,0cyclictest27424-21diskmemload22:11:011
174199230,0cyclictest0-21swapper/100:04:031
1737992323,0cyclictest27424-21diskmemload22:17:330
1737992323,0cyclictest0-21swapper/000:13:310
173799232,21cyclictest62550irq/127-eno100:15:410
1737992322,1cyclictest27424-21diskmemload23:29:510
1737992321,2cyclictest27813-21rm23:46:530
1737992321,1cyclictest0-21swapper/021:35:010
1737992320,2cyclictest22628-21sh21:00:010
1737992320,2cyclictest22628-21sh21:00:010
173799230,2cyclictest783-21snmpd21:43:560
173799230,23cyclictest27424-21diskmemload22:31:200
173799230,23cyclictest27424-21diskmemload22:23:080
173799230,23cyclictest0-21swapper/023:37:180
173799230,23cyclictest0-21swapper/022:14:060
173799230,23cyclictest0-21swapper/000:01:410
173799230,22cyclictest801-21sh21:15:420
173799230,22cyclictest29911-21ssh23:17:360
173799230,1cyclictest9963-21diskstats21:55:460
173799230,1cyclictest30790-21ssh22:46:200
173799230,0cyclictest27424-21diskmemload23:08:200
173799230,0cyclictest0-21swapper/022:36:570
173799230,0cyclictest0-21swapper/000:25:140
1749992222,0cyclictest0-21swapper/300:39:403
1749992221,1cyclictest27424-21diskmemload22:10:223
1749992220,1cyclictest0-21swapper/322:25:143
1749992220,1cyclictest0-21swapper/321:40:023
1749992219,2cyclictest498-21cron20:15:013
174999220,2cyclictest28540-21ssh23:48:073
174999220,22cyclictest0-21swapper/300:25:463
174999220,21cyclictest27424-21diskmemload21:15:503
174999220,21cyclictest17227-21diskstats23:05:453
174999220,21cyclictest11694-21apt-get19:30:433
174999220,21cyclictest1018-21ssh22:50:053
174999220,21cyclictest0-21swapper/323:10:593
174999220,21cyclictest0-21swapper/321:05:443
174999220,1cyclictest9298-21apt-get19:25:423
174999220,1cyclictest8564-21sh00:00:453
174999220,1cyclictest21300-21apt-get19:50:433
1745992222,0cyclictest0-21swapper/223:26:282
1745992222,0cyclictest0-21swapper/222:16:022
1745992222,0cyclictest0-21swapper/222:10:232
1745992222,0cyclictest0-21swapper/221:30:572
1745992222,0cyclictest0-21swapper/200:00:262
1745992221,1cyclictest27424-21diskmemload21:45:142
1745992221,1cyclictest27424-21diskmemload21:45:142
1745992221,1cyclictest0-21swapper/220:45:422
1745992220,1cyclictest4189-21munin-run19:15:012
174599220,21cyclictest605-21avahi-daemon19:50:002
174599220,21cyclictest27424-21diskmemload21:15:412
174599220,21cyclictest27424-21diskmemload00:25:142
174599220,21cyclictest13992sleep222:50:452
174599220,21cyclictest13039-21sendmail-msp20:40:002
174599220,21cyclictest12257-21diskmemload00:35:412
174599220,21cyclictest0-21swapper/222:45:152
174599220,21cyclictest0-21swapper/222:20:072
174599220,21cyclictest0-21swapper/221:50:392
174599220,21cyclictest0-21swapper/221:11:142
174599220,1cyclictest32142-21apt-get23:20:452
174599220,1cyclictest28507-21apt-get20:05:442
174599220,1cyclictest22623-21cron21:00:012
174599220,1cyclictest22623-21cron21:00:012
174599220,1cyclictest0-21swapper/223:50:132
174599220,1cyclictest0-21swapper/223:40:492
1741992222,0cyclictest0-21swapper/120:05:441
1741992222,0cyclictest0-21swapper/100:09:021
1741992221,0cyclictest0-21swapper/123:40:051
1741992220,1cyclictest0-21swapper/122:00:461
1741992220,1cyclictest0-21swapper/121:50:471
174199220,22cyclictest0-21swapper/123:09:381
174199220,22cyclictest0-21swapper/121:32:101
174199220,21cyclictest1-21systemd21:10:011
174199220,21cyclictest0-21swapper/121:10:311
174199220,1cyclictest1897-21ssh00:25:371
174199220,1cyclictest0-21swapper/123:46:191
174199220,1cyclictest0-21swapper/122:44:491
174199220,1cyclictest0-21swapper/122:32:011
174199220,1cyclictest0-21swapper/121:49:431
174199220,1cyclictest0-21swapper/121:49:431
174199220,0cyclictest0-21swapper/123:00:451
174199220,0cyclictest0-21swapper/120:55:411
174199220,0cyclictest0-21swapper/120:55:411
1737992222,0cyclictest0-21swapper/022:00:210
1737992222,0cyclictest0-21swapper/021:35:140
1737992222,0cyclictest0-21swapper/021:10:270
1737992221,1cyclictest4267-21ssh21:50:390
1737992221,1cyclictest0-21swapper/023:13:370
1737992221,1cyclictest0-21swapper/021:25:090
173799220,22cyclictest0-21swapper/023:54:080
173799220,22cyclictest0-21swapper/021:21:190
173799220,21cyclictest0-21swapper/019:40:440
173799220,20cyclictest27424-21diskmemload23:55:490
173799220,1cyclictest9556-21sh22:25:530
173799220,1cyclictest3571-21ssh22:51:230
173799220,1cyclictest31559-21ssh21:45:370
173799220,1cyclictest31559-21ssh21:45:370
173799220,1cyclictest23186-21ssh22:40:040
173799220,1cyclictest14282-21sh00:36:130
173799220,1cyclictest10724-21ssh22:59:400
173799220,0cyclictest0-21swapper/022:08:380
173799220,0cyclictest0-21swapper/000:30:510
174599210,20cyclictest5757-21apt-get20:25:432
174599210,0cyclictest3310-21apt-get20:20:442
174599210,0cyclictest0-21swapper/219:20:432
1741992119,1cyclictest6585-21turbostat19:20:011
1741992119,1cyclictest28859-21rm21:41:421
174199210,21cyclictest27424-21diskmemload22:25:011
174199210,20cyclictest0-21swapper/120:35:441
174199210,0cyclictest0-21swapper/120:25:431
173799210,21cyclictest0-21swapper/020:50:430
173799210,20cyclictest606-21dbus-daemon19:15:010
173799210,20cyclictest0-21swapper/020:20:010
173799210,0cyclictest0-21swapper/020:10:450
1749992020,0cyclictest0-21swapper/320:15:423
1749992020,0cyclictest0-21swapper/320:00:443
1749992018,1cyclictest6591-21perf19:20:013
174999200,20cyclictest0-21swapper/320:20:443
174999200,0cyclictest0-21swapper/320:25:433
1745992020,0cyclictest0-21swapper/219:55:442
1745992020,0cyclictest0-21swapper/219:35:452
1745992018,1cyclictest4483-21apt-get19:15:422
1741992020,0cyclictest0-21swapper/119:50:431
174199200,17cyclictest0-21swapper/120:30:431
1737992020,0cyclictest0-21swapper/020:20:440
1737992020,0cyclictest0-21swapper/020:05:440
1737992020,0cyclictest0-21swapper/019:50:430
1737992020,0cyclictest0-21swapper/019:35:450
1737992019,1cyclictest20989-21perf19:50:000
1737992019,1cyclictest15445-21grep20:45:000
173799200,20cyclictest0-21swapper/020:35:450
173799200,20cyclictest0-21swapper/020:25:430
173799200,20cyclictest0-21swapper/019:55:440
1749991919,0cyclictest20546-21apt-get20:55:413
1749991919,0cyclictest20546-21apt-get20:55:413
1749991919,0cyclictest0-21swapper/320:05:443
1749991919,0cyclictest0-21swapper/319:10:453
1749991918,1cyclictest8551-21apt-get20:30:433
1741991919,0cyclictest26113-21apt-get20:00:441
1741991919,0cyclictest2037-21apt-get19:10:451
1741991919,0cyclictest0-21swapper/121:00:441
1737991919,0cyclictest6904-21apt-get19:20:430
1737991917,1cyclictest6573-21grep19:20:010
1749991817,0cyclictest0-21swapper/320:45:423
1745991818,0cyclictest0-21swapper/220:50:432
1745991818,0cyclictest0-21swapper/219:50:432
1745991816,1cyclictest18606-21turbostat19:45:002
1741991818,0cyclictest0-21swapper/120:20:441
1741991818,0cyclictest0-21swapper/120:15:431
1741991818,0cyclictest0-21swapper/119:30:431
1741991817,1cyclictest0-21swapper/119:25:421
1737991817,1cyclictest0-21swapper/020:45:420
1749991717,0cyclictest0-21swapper/319:20:433
1749991716,0cyclictest18153-21apt-get20:50:443
1745991717,0cyclictest27431-21ssh21:10:012
1745991717,0cyclictest0-21swapper/219:30:432
174199170,16cyclictest15756-21apt-get20:45:421
1741991616,0cyclictest0-21swapper/120:50:441
1741991616,0cyclictest0-21swapper/120:10:451
1741991614,1cyclictest9004-21munin-run19:25:011
1741991614,1cyclictest23696-21apt-get19:55:441
174199160,15cyclictest16046-21runrttasks19:38:431
173799160,15cyclictest12333-21tail19:30:450
174999150,1cyclictest0-21swapper/319:45:523
174999150,14cyclictest23954-21cstates19:55:453
1745991515,0cyclictest0-21swapper/221:00:442
1745991515,0cyclictest0-21swapper/219:25:422
1745991513,1cyclictest0-21swapper/220:15:012
173799150,15cyclictest28079-21chrt20:03:540
174999140,7cyclictest0-21swapper/320:35:443
174999140,14cyclictest0-21swapper/321:00:423
174999140,14cyclictest0-21swapper/319:41:343
174999140,0cyclictest0-21swapper/320:40:083
174999140,0cyclictest0-21swapper/319:35:093
174599140,14cyclictest0-21swapper/220:30:292
1741991414,0cyclictest0-21swapper/120:40:161
1741991414,0cyclictest0-21swapper/119:45:041
1741991414,0cyclictest0-21swapper/119:40:131
1737991414,0cyclictest0-21swapper/021:00:050
1737991414,0cyclictest0-21swapper/020:30:010
173799140,0cyclictest0-21swapper/021:05:140
173799140,0cyclictest0-21swapper/019:25:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional