You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 13:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Apr 25, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20235991226,99cyclictest0-21swapper/005:10:540
272362970,5sleep127232-21cron04:50:481
2023799864,36cyclictest7850irq/56-dwc_otg_08:41:032
64332790,3sleep06436-21cut09:16:090
20235997964,5cyclictest0-21swapper/007:51:010
303262780,4sleep330325-21ssh09:06:063
2023599759,63cyclictest26090-21missed_timers08:21:100
2023599755,14cyclictest0-21swapper/009:36:030
20236997330,11cyclictest1876-21unixbench-2d07:51:171
20236997329,21cyclictest29292-21/usr/sbin/munin04:55:581
2023599737,63cyclictest0-21swapper/007:21:020
2023599731,62cyclictest0-21swapper/008:31:590
2023599731,62cyclictest0-21swapper/008:31:590
88452720,1sleep10-21swapper/108:00:531
88452720,1sleep10-21swapper/108:00:531
311822720,2sleep20-21swapper/207:47:542
230332720,2sleep10-21swapper/104:36:151
2023599713,56cyclictest0-21swapper/005:25:480
20235997119,28cyclictest27767-21latency_hist06:20:490
268972700,2sleep30-21swapper/307:03:503
2023899706,4cyclictest0-21swapper/309:16:013
20236997028,26cyclictest11919-21ssh08:43:201
2023599709,42cyclictest31369-21fschecks_count05:01:050
2023599706,39cyclictest19311-21kernelversion05:56:080
20235997023,28cyclictest29536-21latency_hist06:25:490
2023599698,32cyclictest11-21rcu_preempt09:01:020
2023599686,44cyclictest11179-21ssh09:21:160
2023599681,46cyclictest0-21swapper/008:26:010
20236996728,17cyclictest23121-21ssh06:59:401
2023599679,25cyclictest0-21swapper/009:31:010
2023599678,40cyclictest28700-21latency_hist07:45:490
2023599675,8cyclictest3816-21munin-run07:15:480
2023599673,37cyclictest0-21swapper/005:06:000
2023599673,37cyclictest0-21swapper/005:06:000
75292660,1sleep10-21swapper/107:58:521
2023899661,7cyclictest30826-21/usr/sbin/munin09:46:153
20236996630,11cyclictest18197-21/usr/sbin/munin08:51:061
20235996615,47cyclictest0-21swapper/004:30:490
116932660,5sleep32023899cyclictest08:42:553
20236996528,21cyclictest16491-21rm08:10:121
2023599657,46cyclictest1039-21ssh06:32:170
2023599653,42cyclictest17671-21/usr/sbin/munin08:51:110
316362640,3sleep3381ktimersoftd/307:48:433
20236996431,11cyclictest14904-21/usr/sbin/munin09:26:111
20236996427,19cyclictest29239-21ssh07:46:051
20236996420,15cyclictest28569-21sshd06:21:111
2023599642,34cyclictest0-21swapper/008:01:000
2023599642,34cyclictest0-21swapper/008:01:000
20235996412,13cyclictest23206-21unixbench_singl04:36:170
20235996410,42cyclictest0-21swapper/007:48:460
20236996330,18cyclictest4851-21ls07:16:101
2023599639,51cyclictest4167-21ssh09:51:520
2023599638,40cyclictest0-21swapper/007:59:200
2023599632,48cyclictest0-21swapper/005:26:090
2023599632,38cyclictest10701-21grep08:41:170
20235996311,7cyclictest12061-21grep06:46:060
20235996311,7cyclictest12061-21grep06:46:060
20235996310,30cyclictest4000-21ssh09:13:000
2023899622,12cyclictest0-21swapper/309:20:593
20236996229,22cyclictest19619-21ssh08:13:211
20236996229,15cyclictest2975-21ssh08:32:021
20236996229,15cyclictest2975-21ssh08:32:021
20236996229,11cyclictest18288-21sh07:32:081
20236996229,11cyclictest18288-21sh07:32:081
20236996228,12cyclictest11687-21/usr/sbin/munin06:45:591
20236996228,12cyclictest11687-21/usr/sbin/munin06:45:591
20236996227,21cyclictest7535-21ssh08:37:511
20236996227,20cyclictest18740-21ssh06:54:121
20236996227,12cyclictest11245-21ssh07:24:191
2023599624,47cyclictest0-21swapper/009:00:320
2023599623,41cyclictest0-21swapper/007:37:390
2023599622,49cyclictest0-21swapper/007:26:460
2023899617,39cyclictest0-21swapper/305:51:063
2023799612,51cyclictest0-21swapper/207:40:542
20236996130,12cyclictest15405-21ssh07:29:251
20236996130,12cyclictest11518-21ssh09:21:381
20236996121,16cyclictest6194-21fschecks_count09:16:071
2023599619,35cyclictest30861-21rm09:46:020
2023599618,38cyclictest23761-21sh08:18:230
2023599615,39cyclictest15928-21sh06:51:030
20235996110,36cyclictest26887-21sh07:03:490
2023899607,5cyclictest7950irq/56-dwc_otg_05:26:023
2023899607,4cyclictest7750irq/56-dwc_otg08:16:113
20236996032,11cyclictest9737-21ssh06:42:511
20236996027,20cyclictest5537-21ssh09:15:481
20236996023,16cyclictest31149-21cut05:01:021
20236996022,11cyclictest7750irq/56-dwc_otg08:46:071
20236996021,13cyclictest32009-21ssh06:31:061
20236996019,15cyclictest8345-21/usr/sbin/munin05:26:091
20236996018,26cyclictest19650-21cat05:56:131
2023599606,43cyclictest0-21swapper/007:06:120
2023599605,42cyclictest15704-21ssh09:26:460
20235996033,15cyclictest7750irq/56-dwc_otg06:36:170
2023599601,56cyclictest22346-21latency_hist06:05:490
2023599601,34cyclictest14403-21grep05:41:160
2023599601,31cyclictest21033-21latency_hist08:15:490
20238995920,4cyclictest1967-21cpuspeed09:11:033
2023899591,4cyclictest26305-21munin-run09:40:493
20236995930,12cyclictest20422-21ssh09:32:391
20236995929,17cyclictest5502-21ssh09:54:181
20236995929,13cyclictest22852-21ssh07:37:561
20236995924,26cyclictest25959-21ssh08:21:091
20236995924,13cyclictest32529-21rm08:29:201
20236995919,15cyclictest32204-21rm09:46:221
2023599598,37cyclictest0-21swapper/005:46:020
2023599592,44cyclictest0-21swapper/009:44:030
2023599592,36cyclictest0-21swapper/005:36:180
2023599591,24cyclictest17343-21grep05:51:060
20235995910,45cyclictest0-21swapper/007:15:590
104982590,1sleep20-21swapper/209:21:062
2023899587,4cyclictest0-21swapper/307:52:293
2023899584,12cyclictest7950irq/56-dwc_otg_07:10:503
20236995827,17cyclictest31242-21diskmemload07:02:251
20236995818,14cyclictest27308-21ssh07:43:201
20235995812,29cyclictest0-21swapper/008:46:010
20238995719,5cyclictest26773-21ssh08:21:203
2023899571,7cyclictest26710-21ssh09:01:113
2023799573,25cyclictest0-21swapper/209:01:022
20236995729,11cyclictest10092-21ssh10:00:111
20236995729,11cyclictest10092-21ssh10:00:111
20236995728,12cyclictest7319-21taskset05:23:061
20236995719,15cyclictest23944-21ssh08:18:441
20236995718,24cyclictest0-21swapper/109:04:521
2023599578,39cyclictest0-21swapper/009:10:180
2023599578,38cyclictest0-21swapper/006:44:120
2023599575,49cyclictest883450rm08:40:140
20235995739,4cyclictest7850irq/56-dwc_otg_08:10:480
2023599572,40cyclictest21475-21taskset06:56:390
192612570,4sleep319248-21ssh09:31:143
2023899561,16cyclictest0-21swapper/306:43:243
20236995628,17cyclictest21170-21switchtime04:31:141
20236995628,12cyclictest23490-21ssh08:56:551
20236995626,18cyclictest31808-21ssh07:10:151
20236995617,12cyclictest32262-21runrttasks09:07:521
2023599569,39cyclictest5474-21runrttasks05:17:320
2023899556,3cyclictest0-21swapper/306:33:473
2023899556,12cyclictest0-21swapper/307:19:193
2023799552,10cyclictest0-21swapper/205:01:012
20236995517,15cyclictest908-21runrttasks04:53:091
20236995514,16cyclictest7017-21sh06:40:271
2023599552,4cyclictest16518-21ssh07:31:010
2023599552,4cyclictest16518-21ssh07:31:010
20236995424,18cyclictest17011-21munin-node05:51:001
20236995422,11cyclictest26668-21ssh09:40:551
2023599544,47cyclictest9964-21if_err_eth005:31:060
2023599543,36cyclictest23715-21chrt06:07:110
20235995424,18cyclictest26745-21grep04:46:160
2023599541,39cyclictest0-21swapper/006:12:380
2023599541,39cyclictest0-21swapper/006:12:380
20235995410,26cyclictest8355-21ssh09:56:590
20235995410,26cyclictest8355-21ssh09:56:590
2023899536,3cyclictest0-21swapper/307:31:523
2023899536,3cyclictest0-21swapper/307:31:523
20235995311,27cyclictest25042-21grep04:41:170
2023899521,3cyclictest7750irq/56-dwc_otg07:39:193
2023899521,3cyclictest0-21swapper/307:56:553
20236995218,12cyclictest14812-21chrt05:45:061
20236995217,14cyclictest24812-21ssh09:38:071
2023799517,23cyclictest0-21swapper/207:51:032
2023799511,5cyclictest0-21swapper/209:41:022
20236995120,13cyclictest25343-21chrt04:44:361
20236995116,13cyclictest7750irq/56-dwc_otg06:15:501
20236995116,13cyclictest7750irq/56-dwc_otg06:15:501
2023599513,39cyclictest0-21swapper/006:29:070
2023799504,6cyclictest0-21swapper/206:36:022
20236995020,14cyclictest2276-21ssh07:13:011
20237994910,6cyclictest7950irq/56-dwc_otg_07:31:002
20237994910,6cyclictest7950irq/56-dwc_otg_07:31:002
20236994927,11cyclictest3854-21taskset05:14:481
20236994918,9cyclictest0-21swapper/106:01:431
20236994916,13cyclictest23952-21sshd06:10:121
2023799483,12cyclictest0-21swapper/207:56:022
20237994822,5cyclictest17935-21idleruntime-cro09:30:492
308662470,1sleep30-21swapper/305:00:493
2023799477,25cyclictest0-21swapper/205:11:012
20236994722,11cyclictest12178-21/usr/sbin/munin05:36:121
20236994716,10cyclictest0-21swapper/106:26:501
20235994724,13cyclictest0-21swapper/004:51:130
2023799467,23cyclictest0-21swapper/207:21:022
2023699467,27cyclictest0-21swapper/105:45:541
2023699463,5cyclictest0-21swapper/106:16:011
20236994621,10cyclictest5513-21chrt05:18:011
2023599467,31cyclictest0-21swapper/004:55:580
2023799457,25cyclictest0-21swapper/208:26:012
2023799457,24cyclictest0-21swapper/205:35:542
2023799456,24cyclictest0-21swapper/208:36:022
2023799451,12cyclictest0-21swapper/206:51:022
20236994530,12cyclictest2156-21apt-config05:10:491
20236994530,12cyclictest2156-21apt-config05:10:491
20236994516,11cyclictest0-21swapper/105:31:411
2023899444,8cyclictest7950irq/56-dwc_otg_05:31:013
2023799447,23cyclictest0-21swapper/206:26:012
2023799447,23cyclictest0-21swapper/204:51:012
2023899436,34cyclictest908-21runrttasks09:41:093
2023799437,25cyclictest0-21swapper/209:30:542
2023799429,7cyclictest7850irq/56-dwc_otg_05:51:002
2023799427,24cyclictest0-21swapper/205:45:542
2023799424,12cyclictest0-21swapper/205:06:002
2023799424,12cyclictest0-21swapper/205:06:002
2023799421,4cyclictest7750irq/56-dwc_otg08:46:002
199592420,2sleep30-21swapper/308:53:013
2023799417,22cyclictest0-21swapper/205:16:022
20237994110,7cyclictest0-21swapper/208:56:022
2023799407,23cyclictest0-21swapper/205:21:022
2023799404,23cyclictest7750irq/56-dwc_otg08:51:052
20237994010,4cyclictest0-21swapper/209:39:242
300772390,4sleep330082-21kernelversion08:26:083
2023899391,22cyclictest2730-21ls05:11:063
2023799399,4cyclictest0-21swapper/208:07:182
219382380,10sleep27950irq/56-dwc_otg_07:36:182
2023899385,22cyclictest7950irq/56-dwc_otg_08:36:053
20238993811,4cyclictest22739-21sh08:56:143
2023799387,24cyclictest0-21swapper/205:41:012
2023799386,4cyclictest29-21rcuc/208:21:062
2023799384,8cyclictest7950irq/56-dwc_otg_07:06:112
2023799384,21cyclictest7850irq/56-dwc_otg_06:01:022
2023799383,14cyclictest31-21ksoftirqd/209:46:042
20237993820,11cyclictest31-21ksoftirqd/207:16:042
2023799381,27cyclictest9459-21ssh08:01:092
2023799381,27cyclictest9459-21ssh08:01:092
20238993714,4cyclictest21832-21munin-plugin-st04:35:483
20238993713,5cyclictest908-21runrttasks08:05:263
20238993713,5cyclictest908-21runrttasks08:05:263
2023899371,21cyclictest10606-21ls10:00:503
2023899371,21cyclictest10606-21ls10:00:503
20237993715,8cyclictest7950irq/56-dwc_otg_08:16:072
2023799371,28cyclictest7750irq/56-dwc_otg06:56:532
20237993710,7cyclictest31-21ksoftirqd/209:16:042
2023899363,16cyclictest0-21swapper/306:06:023
2023799365,6cyclictest7750irq/56-dwc_otg09:15:492
2023799363,23cyclictest7750irq/56-dwc_otg06:31:492
20237993614,11cyclictest7750irq/56-dwc_otg06:48:192
20237993614,11cyclictest7750irq/56-dwc_otg06:48:192
20237993613,6cyclictest11219-21sh06:45:342
2023799361,23cyclictest8348-21missed_timers05:26:102
2023799361,20cyclictest0-21swapper/207:11:202
20237993611,8cyclictest29-21rcuc/205:56:092
1843523619,7sleep00-21swapper/004:26:020
2023899351,25cyclictest29958-21cut06:26:043
20238993510,5cyclictest0-21swapper/306:45:523
20238993510,5cyclictest0-21swapper/306:45:523
20237993514,5cyclictest29-21rcuc/206:20:492
2023799349,15cyclictest0-21swapper/207:01:052
2023799348,16cyclictest7850irq/56-dwc_otg_04:46:132
2023799346,7cyclictest29-21rcuc/205:31:162
2023799346,20cyclictest7850irq/56-dwc_otg_07:26:062
2023799345,20cyclictest2552-21ssh09:51:022
2023799342,7cyclictest18353-21grep08:11:172
2023799342,5cyclictest1426-21awk09:10:482
2023899331,6cyclictest32262-21unixbench_singl05:01:163
20238993312,6cyclictest908-21runrttasks09:30:503
20238993310,4cyclictest7950irq/56-dwc_otg_04:41:003
2023799336,15cyclictest0-21swapper/209:56:022
2023799336,15cyclictest0-21swapper/209:56:022
2023799332,7cyclictest25025-21unixbench_singl04:41:172
2023799331,9cyclictest301ktimersoftd/204:30:492
20237993314,13cyclictest35282chrt08:33:032
20237993314,13cyclictest35282chrt08:33:032
20237993311,5cyclictest29-21rcuc/206:21:072
2023899328,6cyclictest7850irq/56-dwc_otg_08:11:013
2023899327,7cyclictest37-21rcuc/308:06:203
2023899326,4cyclictest7950irq/56-dwc_otg_04:36:023
2023899326,14cyclictest26176-21ssh07:41:183
2023899325,16cyclictest0-21swapper/307:29:053
2023899323,11cyclictest0-21swapper/307:21:003
20238993225,5cyclictest32573-21wget07:11:033
2023899322,3cyclictest7850irq/56-dwc_otg_06:56:023
20238993211,5cyclictest37-21rcuc/308:46:143
20238993211,10cyclictest0-21swapper/308:33:403
20238993211,10cyclictest0-21swapper/308:33:403
2023799325,15cyclictest7950irq/56-dwc_otg_04:56:042
20237993212,7cyclictest31-21ksoftirqd/204:36:162
2023899317,5cyclictest37-21rcuc/306:36:023
2023899317,4cyclictest21767-21/usr/sbin/munin06:01:153
2023899312,5cyclictest0-21swapper/309:55:503
20238993112,6cyclictest15896-21ntp_kernel_pll_05:46:113
2023899311,20cyclictest19840-21timerwakeupswit05:56:153
2023899311,19cyclictest27658-21ls06:20:483
2023799311,20cyclictest22537-21aten_rbpower_po06:06:042
20238993018,10cyclictest7950irq/56-dwc_otg_06:53:373
2023899301,19cyclictest0-21swapper/304:55:323
2023899299,7cyclictest12451-21timerandwakeup05:36:163
2023899297,5cyclictest7950irq/56-dwc_otg_05:21:053
2023899293,17cyclictest0-21swapper/306:25:483
2023799292,19cyclictest0-21swapper/206:11:002
2023799292,19cyclictest0-21swapper/206:11:002
2023899282,17cyclictest0-21swapper/305:43:063
20238992820,5cyclictest24370-21cut06:11:043
20238992820,5cyclictest24370-21cut06:11:043
20238992720,4cyclictest381ktimersoftd/304:46:023
2023899271,7cyclictest1192-21/usr/sbin/munin05:06:133
2023899271,7cyclictest1192-21/usr/sbin/munin05:06:133
198292274,15sleep20-21swapper/204:29:042
197212275,14sleep10-21swapper/104:27:371
2023899262,6cyclictest7950irq/56-dwc_otg_05:16:093
184332265,15sleep318430-21apt-get04:26:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional