You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-24 - 07:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Wed Jun 24, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1253599270,26cyclictest0-21swapper/222:18:032
1253399260,25cyclictest0-21swapper/022:10:200
12537992525,0cyclictest0-21swapper/322:18:033
12535992525,0cyclictest0-21swapper/223:28:042
12535992520,4cyclictest0-21swapper/222:45:472
1253599250,24cyclictest6166-21diskmemload21:57:522
1253599250,24cyclictest3701-21ssh22:05:102
1253599250,24cyclictest20847-21sed23:45:402
1253599250,24cyclictest0-21swapper/223:15:232
1253599250,24cyclictest0-21swapper/222:56:092
1253599250,22cyclictest12088-21rm22:11:342
12534992523,1cyclictest0-21swapper/100:06:111
12534992520,1cyclictest151rcu_preempt21:16:581
1253499250,24cyclictest1157-21rm23:29:101
1253499250,24cyclictest0-21swapper/123:37:111
12533992523,1cyclictest62550irq/127-eno121:27:560
12533992523,1cyclictest3133-21sh22:32:550
12537992424,0cyclictest0-21swapper/323:15:233
12537992423,1cyclictest9892-21rm22:39:403
12537992423,1cyclictest9892-21rm22:39:403
12537992423,1cyclictest0-21swapper/323:29:103
12537992423,1cyclictest0-21swapper/322:25:303
12537992421,2cyclictest796-21phc2sys22:30:383
1253799240,2cyclictest3000-40gzip00:00:003
12535992423,1cyclictest6166-21diskmemload00:11:302
12535992423,0cyclictest0-21swapper/200:16:272
12535992420,4cyclictest0-21swapper/200:04:362
1253599241,1cyclictest0-21swapper/223:00:282
1253599240,24cyclictest6166-21diskmemload21:23:222
1253599240,24cyclictest0-21swapper/221:54:142
1253599240,24cyclictest0-21swapper/221:54:142
1253599240,23cyclictest0-21swapper/222:25:302
1253599240,23cyclictest0-21swapper/221:34:572
1253599240,23cyclictest0-21swapper/221:25:332
1253599240,23cyclictest0-21swapper/200:24:182
1253599240,1cyclictest8976-21ssh00:32:492
12534992423,1cyclictest6166-21diskmemload21:30:061
12534992422,1cyclictest3517-21ssh00:28:311
12534992422,1cyclictest0-21swapper/122:07:231
12534992422,1cyclictest0-21swapper/121:56:221
12534992420,3cyclictest0-21swapper/122:01:051
1253499241,22cyclictest783-21snmpd21:35:441
1253499240,24cyclictest0-21swapper/123:31:371
1253499240,23cyclictest0-21swapper/123:09:031
1253499240,23cyclictest0-21swapper/121:42:101
1253499240,1cyclictest20844-21ssh23:45:401
1253499240,0cyclictest0-21swapper/123:15:231
1253499240,0cyclictest0-21swapper/100:24:181
12533992422,2cyclictest16164-21ssh00:39:570
1253399240,23cyclictest6166-21diskmemload21:40:090
1253399240,23cyclictest24404-21ssh22:23:070
1253399240,1cyclictest0-21swapper/022:51:390
12537992323,0cyclictest0-21swapper/323:22:013
12537992323,0cyclictest0-21swapper/323:22:013
12537992323,0cyclictest0-21swapper/322:05:103
12537992323,0cyclictest0-21swapper/321:57:523
12537992320,2cyclictest20207-21ssh21:21:123
12537992320,2cyclictest18491-21ssh22:46:063
12537992320,2cyclictest11384-21sh00:06:073
1253799230,2cyclictest15483-21sh21:17:223
1253799230,2cyclictest14594-21rm00:10:093
1253799230,23cyclictest6166-21diskmemload22:20:253
1253799230,23cyclictest6166-21diskmemload00:21:403
1253799230,23cyclictest25171-21sh22:52:353
1253799230,23cyclictest0-21swapper/323:53:183
1253799230,23cyclictest0-21swapper/322:40:233
1253799230,23cyclictest0-21swapper/322:02:113
1253799230,23cyclictest0-21swapper/321:50:393
1253799230,23cyclictest0-21swapper/321:50:393
1253799230,23cyclictest0-21swapper/321:36:163
1253799230,23cyclictest0-21swapper/321:32:073
1253799230,23cyclictest0-21swapper/300:03:593
1253799230,22cyclictest9979-21sh23:07:343
1253799230,22cyclictest5506-21ssh00:30:223
1253799230,22cyclictest11713-21ssh22:11:053
1253799230,22cyclictest11552-21cat00:35:303
1253799230,22cyclictest0-21swapper/321:45:263
1253799230,0cyclictest6166-21diskmemload23:43:203
1253799230,0cyclictest6166-21diskmemload23:35:163
1253799230,0cyclictest0-21swapper/321:42:483
12535992323,0cyclictest6166-21diskmemload22:20:052
12535992323,0cyclictest0-21swapper/222:00:412
12535992323,0cyclictest0-21swapper/200:06:112
12535992319,3cyclictest0-21swapper/223:41:192
1253599230,23cyclictest0-21swapper/223:37:112
1253599230,23cyclictest0-21swapper/223:22:012
1253599230,23cyclictest0-21swapper/223:22:012
1253599230,22cyclictest7883-21sh22:36:502
1253599230,22cyclictest7883-21sh22:36:502
1253599230,22cyclictest31761-21sh22:30:112
1253599230,22cyclictest1891-21ssh00:26:252
1253599230,0cyclictest0-21swapper/223:54:202
1253599230,0cyclictest0-21swapper/223:14:342
1253599230,0cyclictest0-21swapper/222:51:402
12534992323,0cyclictest0-21swapper/123:59:341
12534992323,0cyclictest0-21swapper/100:30:291
12534992322,1cyclictest6166-21diskmemload00:14:381
12534992322,1cyclictest0-21swapper/123:20:031
12534992322,1cyclictest0-21swapper/123:20:031
12534992321,1cyclictest5847-21taskset23:05:001
12534992321,1cyclictest0-21swapper/122:57:301
12534992321,1cyclictest0-21swapper/122:16:111
12534992321,1cyclictest0-21swapper/121:20:331
12534992320,2cyclictest15166-21wc23:40:361
1253499230,23cyclictest6166-21diskmemload22:53:071
1253499230,23cyclictest0-21swapper/123:13:071
1253499230,23cyclictest0-21swapper/122:10:381
1253499230,22cyclictest9716-21sh22:39:261
1253499230,22cyclictest9716-21sh22:39:261
1253499230,22cyclictest774-21runrttasks21:11:391
1253499230,22cyclictest13475-21sh00:36:151
1253499230,22cyclictest0-21swapper/122:30:381
1253499230,22cyclictest0-21swapper/122:25:301
1253499230,22cyclictest0-21swapper/121:50:471
1253499230,22cyclictest0-21swapper/121:50:471
1253499230,22cyclictest0-21swapper/100:17:001
1253499230,0cyclictest6166-21diskmemload22:48:071
1253499230,0cyclictest0-21swapper/121:25:331
12533992323,0cyclictest6166-21diskmemload21:54:110
12533992323,0cyclictest6166-21diskmemload21:54:110
12533992323,0cyclictest0-21swapper/021:34:240
12533992323,0cyclictest0-21swapper/021:16:280
12533992322,1cyclictest62550irq/127-eno123:46:350
12533992322,1cyclictest13730-21ssh22:42:060
12533992321,1cyclictest3657-21ssh22:05:070
12533992321,1cyclictest0-21swapper/023:51:370
12533992321,1cyclictest0-21swapper/000:32:050
1253399230,23cyclictest0-21swapper/022:55:590
1253399230,23cyclictest0-21swapper/022:02:470
1253399230,22cyclictest6180-21ssh00:01:390
1253399230,22cyclictest6166-21diskmemload23:00:320
1253399230,22cyclictest15931-21/usr/sbin/munin22:15:320
1253399230,22cyclictest13095-21runrttasks00:08:280
1253399230,22cyclictest0-21swapper/021:49:070
1253399230,22cyclictest0-21swapper/000:25:340
1253399230,1cyclictest6199-21sh21:37:540
1253399230,1cyclictest0-21swapper/021:24:220
1253399230,0cyclictest0-21swapper/023:57:190
1253399230,0cyclictest0-21swapper/023:24:000
1253399230,0cyclictest0-21swapper/023:24:000
1253399230,0cyclictest0-21swapper/000:23:570
12537992222,0cyclictest0-21swapper/323:31:113
12537992222,0cyclictest0-21swapper/321:25:333
12537992222,0cyclictest0-21swapper/300:16:303
1253799220,21cyclictest8841-21ssh21:10:503
1253799220,21cyclictest29236-21apt-get20:50:293
1253799220,21cyclictest12340-21apt-get23:10:303
1253799220,21cyclictest0-21swapper/300:25:023
1253799220,1cyclictest31131-21ssh22:57:583
1253799220,1cyclictest0-21swapper/323:00:283
12535992222,0cyclictest0-21swapper/221:15:272
12535992221,1cyclictest8445-21apt-get21:40:282
12535992220,1cyclictest30288-21apt-get23:55:302
12535992220,1cyclictest2812-21apt-get19:55:272
12535992220,1cyclictest12786-21timerwakeupswit00:35:382
1253599220,22cyclictest8062-21ssh21:10:372
1253599220,21cyclictest5828-21apt-get23:05:022
1253599220,21cyclictest2444-21ssh21:35:082
1253599220,21cyclictest16008-21sh21:45:562
1253599220,1cyclictest2191-21sh23:30:082
12534992220,1cyclictest7622-21apt-get20:05:281
1253499220,21cyclictest0-21swapper/123:51:121
1253499220,21cyclictest0-21swapper/122:40:291
1253499220,21cyclictest0-21swapper/121:45:011
1253499220,21cyclictest0-21swapper/120:40:281
1253499220,21cyclictest0-21swapper/100:00:301
1253499220,0cyclictest0-21swapper/122:20:331
12533992222,0cyclictest0-21swapper/000:15:440
12533992221,1cyclictest6166-21diskmemload23:44:350
12533992221,1cyclictest16391-21ssh23:13:430
12533992221,1cyclictest0-21swapper/022:35:270
12533992221,1cyclictest0-21swapper/022:35:270
12533992220,1cyclictest0-21swapper/021:10:340
12533992220,1cyclictest0-21swapper/019:45:010
1253399220,22cyclictest5349-21rm23:32:110
1253399220,22cyclictest0-21swapper/000:10:150
1253399220,21cyclictest7980-21apt-get23:35:280
1253399220,21cyclictest0-21swapper/021:55:020
1253399220,1cyclictest0-21swapper/023:28:100
1253399220,1cyclictest0-21swapper/023:17:210
1253399220,0cyclictest26375-21ssh22:25:310
1253399220,0cyclictest0-21swapper/022:46:150
12537992120,1cyclictest24446-21apt-get20:40:283
1253799210,21cyclictest0-21swapper/320:10:293
1253799210,20cyclictest0-21swapper/320:55:303
1253799210,0cyclictest6166-21diskmemload23:45:383
1253799210,0cyclictest17234-21apt-get20:25:273
1253799210,0cyclictest0-21swapper/320:00:293
12535992121,0cyclictest0-21swapper/220:25:282
12535992119,1cyclictest19632-21apt-get20:30:292
1253599210,20cyclictest0-21swapper/220:40:282
1253599210,1cyclictest31686-21apt-get20:55:302
1253599210,1cyclictest0-21swapper/219:15:022
12534992119,1cyclictest27987-21apt-get19:40:281
12534992119,1cyclictest10018-21apt-get20:10:281
1253499210,21cyclictest0-21swapper/119:22:231
1253499210,20cyclictest0-21swapper/120:30:291
1253499210,20cyclictest0-21swapper/120:25:271
12533992121,0cyclictest0-21swapper/020:15:260
12533992121,0cyclictest0-21swapper/019:20:270
12533992120,1cyclictest14891-21apt-get19:15:020
12533992120,0cyclictest1682-21apt-get21:00:280
12533992119,1cyclictest30380-21apt-get19:45:270
1253399210,20cyclictest0-21swapper/020:55:310
1253399210,0cyclictest0-21swapper/020:10:290
1253799200,20cyclictest0-21swapper/319:15:023
1253799200,20cyclictest0-21swapper/319:10:283
1253799200,0cyclictest0-21swapper/320:15:253
12535992020,0cyclictest0-21swapper/220:50:292
12535992020,0cyclictest0-21swapper/219:25:292
12535992019,1cyclictest22030-21apt-get20:35:292
12535992017,2cyclictest313-21systemd-journal19:15:002
12534992020,0cyclictest0-21swapper/120:45:281
12534992020,0cyclictest0-21swapper/120:35:291
12534992019,0cyclictest0-21swapper/120:20:291
12534992018,1cyclictest22869-21cron19:30:001
12534992017,2cyclictest17693-21cron19:20:001
1253499200,0cyclictest0-21swapper/120:15:251
1253399200,0cyclictest0-21swapper/020:35:290
12537991919,0cyclictest0-21swapper/319:25:293
12537991918,0cyclictest0-21swapper/320:05:283
12535991919,0cyclictest0-21swapper/220:45:282
12535991919,0cyclictest0-21swapper/220:05:282
12534991919,0cyclictest0-21swapper/119:55:271
12534991918,1cyclictest1385-21basename21:00:001
12534991917,1cyclictest14906-21dpkg19:15:001
12534991917,1cyclictest0-21swapper/120:50:291
12533991919,0cyclictest0-21swapper/019:30:270
12533991919,0cyclictest0-21swapper/019:25:290
12533991918,1cyclictest0-21swapper/019:55:270
12533991917,1cyclictest28952-21cat20:50:000
1253399190,19cyclictest0-21swapper/020:20:300
12537991818,0cyclictest0-21swapper/319:55:273
12537991817,0cyclictest0-21swapper/320:30:293
12537991816,1cyclictest14843-21apt-get20:20:293
1253799180,18cyclictest0-21swapper/321:10:013
12535991818,0cyclictest0-21swapper/220:10:282
12535991818,0cyclictest0-21swapper/219:40:282
1253599180,3cyclictest24177-21cut19:30:322
1253599180,0cyclictest0-21swapper/221:10:012
12534991818,0cyclictest0-21swapper/120:00:291
12534991817,0cyclictest0-21swapper/119:35:291
12533991817,1cyclictest5230-21apt-get20:00:290
12537991717,0cyclictest0-21swapper/320:45:283
12537991717,0cyclictest0-21swapper/320:35:293
12537991715,1cyclictest25571-21apt-get19:35:293
12535991717,0cyclictest0-21swapper/220:20:292
12535991716,0cyclictest5230-21apt-get20:00:292
12534991715,2cyclictest2522-21idleruntime-cro19:55:001
12533991715,1cyclictest6192-21latency_hist21:10:010
12533991715,1cyclictest2513-21dump-pmu-power19:55:000
12533991715,1cyclictest1-21systemd20:55:000
1253399170,16cyclictest14902-21munin-run19:15:000
12537991616,0cyclictest0-21swapper/319:40:283
12537991615,1cyclictest0-21swapper/319:20:023
12535991616,0cyclictest0-21swapper/219:35:292
12535991616,0cyclictest0-21swapper/219:20:022
12535991614,1cyclictest31482-21ntp_states19:45:332
12535991614,1cyclictest12410-21apt-get20:15:252
1253499160,15cyclictest0-21swapper/121:10:011
12533991614,1cyclictest25934-21expr19:35:300
12537991513,1cyclictest32546-21awk19:50:013
12537991414,0cyclictest0-21swapper/319:30:323
12537991412,1cyclictest2520-21perf19:55:003
1253799140,14cyclictest0-21swapper/321:01:043
1253599140,14cyclictest0-21swapper/219:50:322
12534991414,0cyclictest0-21swapper/119:30:321
1253499140,0cyclictest0-21swapper/121:04:081
12533991412,1cyclictest21711-21cron20:35:010
1253399140,14cyclictest0-21swapper/020:25:280
1253399140,14cyclictest0-21swapper/020:06:000
1253399140,0cyclictest0-21swapper/020:40:320
12535991313,0cyclictest0-21swapper/221:02:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional