You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-22 - 12:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Tue Sep 22, 2026 00:45:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
745847999388,3cyclictest0-21swapper/023:30:020
74584799891,84cyclictest771487-21ssh21:50:460
74584799891,84cyclictest771487-21ssh21:50:460
74584799882,81cyclictest819836-21ssh23:33:540
74584799882,81cyclictest819836-21ssh23:33:540
745847998630,52cyclictest844458-21ssh00:25:500
745847998231,32cyclictest764719-21ssh21:36:280
74584799821,65cyclictest830722-21ssh23:56:510
745847997923,52cyclictest782629-21ssh22:14:460
745847997873,3cyclictest0-21swapper/023:25:030
745847997873,3cyclictest0-21swapper/023:25:030
745847997730,30cyclictest774975-21ssh21:58:260
745847997729,31cyclictest799057-21ssh22:49:430
745847997728,45cyclictest767279-21ssh21:41:520
745847997723,34cyclictest790313-21ssh22:30:440
74584799771,55cyclictest836400-21ssh00:09:030
745847997713,43cyclictest16850irq/134-800000000:20:120
74585099754,67cyclictest848752-21ssh00:35:011
745847997531,27cyclictest789770-21ssh22:29:540
745847997531,27cyclictest789770-21ssh22:29:530
74584799751,55cyclictest786070-21ssh22:21:460
74584799751,55cyclictest786070-21ssh22:21:450
745847997511,13cyclictest802075-21ssh22:55:520
745847997429,27cyclictest752411-21ssh21:10:220
745847997429,27cyclictest752411-21ssh21:10:210
745847997428,30cyclictest757598-21ssh21:21:210
745847997423,47cyclictest777292-21ssh22:03:130
745847997414,55cyclictest16850irq/134-800000021:48:000
745847997327,42cyclictest805769-21ssh23:03:570
745847997319,37cyclictest763586-21ssh21:34:200
745847997230,38cyclictest826170-21ssh23:47:120
745847997128,26cyclictest799735-21ssh22:50:480
74584799712,7cyclictest750453-21munin-run20:40:010
74584799712,6cyclictest748347-21sshd19:59:280
745847997118,37cyclictest783551-21ssh22:16:300
745847997118,37cyclictest16850irq/134-800000023:11:590
745850996927,38cyclictest831452-21sh23:58:311
745847996921,31cyclictest759609-21ssh21:25:370
745847996829,36cyclictest778305-21ssh22:05:040
745847996829,35cyclictest821666-21ssh23:37:450
745847996820,31cyclictest823959-21rm23:42:300
745850996727,35cyclictest850959-21latency_hist00:40:031
745850996726,5cyclictest809342-21sh23:11:261
745847996754,7cyclictest0-21swapper/019:10:010
745847996728,35cyclictest0-21swapper/000:15:560
745847996724,26cyclictest807418-21ssh23:07:180
745847996717,32cyclictest847408-21ssh00:32:120
74584799653,7cyclictest1-21systemd21:20:010
745847996516,33cyclictest834374-21ssh00:04:480
745850996427,31cyclictest773909-21sh21:55:511
74584799645,12cyclictest795017-21ssh22:40:420
745847996417,31cyclictest16850irq/134-800000022:35:470
745857996341,17cyclictest1-21systemd20:25:013
745850996325,5cyclictest763610-21sh21:34:231
745850996322,6cyclictest309-21in:imuxsock19:45:011
74584799637,6cyclictest16850irq/134-800000023:18:120
74584799633,15cyclictest829706-21munin-run23:55:000
74584799632,14cyclictest747964-21latency_hist19:50:020
745850996221,5cyclictest794960-21sh22:40:351
74584799622,6cyclictest750166-21sshd20:32:280
74584799621,11cyclictest750703-21sshd20:43:290
745850996121,5cyclictest808241-21sh23:09:221
74584799612,17cyclictest849111-21ssh00:35:470
74584799611,8cyclictest748076-21sshd19:50:290
745850996025,4cyclictest3467562-21sshd20:24:281
745850996019,4cyclictest762395-21kworker/u8:221:45:021
745850996017,5cyclictest747217-21cat19:35:051
745850996017,5cyclictest747217-21cat19:35:051
745850995928,27cyclictest23-21ksoftirqd/121:20:021
745850995927,28cyclictest754564-21sh21:15:021
745850995927,28cyclictest754564-21sh21:15:021
745850995917,9cyclictest752113-21cat21:10:011
74584799591,12cyclictest364-21snmpd19:29:230
745855995841,12cyclictest275-21dbus-daemon21:05:012
74585099584,16cyclictest787456-21munin-run22:25:001
74585099584,16cyclictest787456-21munin-run22:25:001
745850995823,4cyclictest760186-21sh21:27:031
745850995822,9cyclictest23-21ksoftirqd/121:21:541
745850995816,10cyclictest762623-1kworker/u9:221:38:121
745850995814,11cyclictest3467602-21systemd-journal19:25:011
74584799583,8cyclictest751772-21munin-run21:05:010
74584799582,9cyclictest746266-21sshd19:17:280
745847995812,42cyclictest0-21swapper/000:10:320
74584799581,14cyclictest748874-21sshd20:08:280
74584799581,14cyclictest748874-21sshd20:08:280
745847995729,25cyclictest747642-21cut19:45:020
74584799571,11cyclictest746550-21sshd19:23:280
745850995619,8cyclictest23-21ksoftirqd/123:42:551
745850995619,10cyclictest23-21ksoftirqd/100:28:271
74584799565,11cyclictest275-21dbus-daemon20:20:010
74584799562,13cyclictest364-21snmpd19:32:140
74584799562,13cyclictest364-21snmpd19:32:140
74584799562,10cyclictest364-21snmpd21:10:020
745855995547,5cyclictest508491-21kworker/2:220:35:022
745850995526,26cyclictest821431-21sh23:37:041
745850995517,8cyclictest23-21ksoftirqd/122:33:111
745850995512,36cyclictest0-21swapper/100:20:011
74584799553,6cyclictest749939-21munin-run20:30:000
74584799552,7cyclictest364-21snmpd20:56:550
745857995434,13cyclictest275-21dbus-daemon20:15:003
745857995431,17cyclictest275-21dbus-daemon22:15:023
745855995443,8cyclictest508491-21kworker/2:221:43:122
745850995420,5cyclictest817318-21sh23:28:391
745850995418,7cyclictest23-21ksoftirqd/120:37:151
74584799542,7cyclictest747318-21cron19:40:000
74584799542,13cyclictest748657-21munin-run20:05:010
74584799541,11cyclictest749116-21sshd20:11:280
745855995345,5cyclictest749949-21kworker/2:322:45:222
74585099534,12cyclictest748353-21modprobe20:00:001
745850995324,4cyclictest819391-21sh23:32:571
745850995324,4cyclictest819391-21sh23:32:571
745850995322,4cyclictest746298-21latency_hist19:20:031
745850995320,5cyclictest799688-21sh22:50:421
745847995322,27cyclictest19350irq/34-mmc120:50:010
745847995322,27cyclictest19350irq/34-mmc120:50:010
74584799531,2cyclictest0-21swapper/019:15:000
745855995244,5cyclictest749949-21kworker/2:322:39:152
745855995243,5cyclictest794544-21kworker/2:023:05:092
745855995242,6cyclictest749949-21kworker/2:322:25:062
745855995242,6cyclictest749949-21kworker/2:322:25:062
74585099524,30cyclictest751446-21kworker/u8:021:45:031
745850995219,8cyclictest23-21ksoftirqd/100:11:581
745850995216,9cyclictest793219-1kworker/u9:022:38:291
745850995215,9cyclictest827534-21sed23:50:021
745847995231,17cyclictest749685-21latency_hist20:25:030
74584799521,12cyclictest751195-21sshd20:51:290
745857995133,14cyclictest1-21systemd20:55:003
745855995143,5cyclictest794544-21kworker/2:023:47:222
745855995143,5cyclictest794544-21kworker/2:023:20:002
745855995143,5cyclictest749949-21kworker/2:323:01:412
745855995142,6cyclictest803878-21kworker/2:200:38:362
745855995142,6cyclictest508491-21kworker/2:221:38:082
745855995142,5cyclictest803878-21kworker/2:200:23:582
745850995118,8cyclictest23-21ksoftirqd/100:04:141
745850995116,10cyclictest23-21ksoftirqd/123:03:441
745855995042,5cyclictest803878-21kworker/2:223:58:252
745855995042,5cyclictest803878-21kworker/2:200:12:092
745855995042,5cyclictest508491-21kworker/2:221:46:132
745855995041,6cyclictest749949-21kworker/2:322:00:002
745850995021,4cyclictest749955-21latency_hist20:30:011
745850995018,6cyclictest752012-21diskmemload22:20:011
745850995014,13cyclictest275-21dbus-daemon20:20:011
745850995010,5cyclictest131rcu_preempt22:05:301
74585799492,17cyclictest785071-21modprobe22:19:593
745855994941,5cyclictest803878-21kworker/2:200:08:362
745855994941,5cyclictest508491-21kworker/2:222:33:092
745855994940,5cyclictest794544-21kworker/2:023:25:162
745855994940,5cyclictest508491-21kworker/2:222:14:562
745855994929,14cyclictest0-21swapper/223:31:342
745855994929,14cyclictest0-21swapper/223:31:342
745850994921,4cyclictest782259-21sh22:13:551
745850994911,6cyclictest131rcu_preempt23:53:321
745857994835,10cyclictest831017-21sshd23:57:293
745855994840,6cyclictest749949-21kworker/2:321:17:192
745855994840,5cyclictest803878-21kworker/2:200:18:082
745855994840,5cyclictest508491-21kworker/2:221:33:002
74585099489,6cyclictest131rcu_preempt00:09:121
745850994819,8cyclictest23-21ksoftirqd/121:05:021
745850994818,8cyclictest23-21ksoftirqd/123:18:081
745850994818,6cyclictest23-21ksoftirqd/120:45:021
745850994815,8cyclictest0-21swapper/121:51:201
745850994815,8cyclictest0-21swapper/121:51:201
745857994732,9cyclictest794498-21munin-plugin-st22:40:003
745855994739,5cyclictest738513-21kworker/2:120:20:022
745855994736,6cyclictest749949-21kworker/2:321:11:382
745855994736,6cyclictest749949-21kworker/2:321:11:382
745855994734,10cyclictest3467602-21systemd-journal00:00:292
74585599473,3cyclictest131rcu_preempt22:16:062
745857994627,14cyclictest1-21systemd22:50:003
74585799461,26cyclictest839363-21ssh00:15:043
74585599464,3cyclictest131rcu_preempt22:02:552
745855994638,5cyclictest803878-21kworker/2:200:33:242
745855994638,5cyclictest794544-21kworker/2:023:21:082
745855994638,5cyclictest794544-21kworker/2:023:21:072
745855994638,5cyclictest749949-21kworker/2:321:53:502
745855994638,5cyclictest749949-21kworker/2:321:53:502
745855994637,6cyclictest738513-21kworker/2:119:47:242
745857994534,9cyclictest37-21ksoftirqd/321:16:363
745857994530,10cyclictest3467602-21systemd-journal20:40:033
745857994529,13cyclictest37-21ksoftirqd/321:13:263
745857994529,13cyclictest37-21ksoftirqd/321:13:263
745857994529,13cyclictest1-21systemd19:35:003
745857994529,13cyclictest1-21systemd19:35:003
745857994527,9cyclictest748365-21sh20:00:003
745857994520,21cyclictest37-21ksoftirqd/323:05:023
745855994537,5cyclictest738513-21kworker/2:119:44:562
745855994537,5cyclictest508491-21kworker/2:220:25:282
745855994535,6cyclictest30-21ksoftirqd/222:22:572
745855994535,6cyclictest30-21ksoftirqd/222:22:572
745855994533,9cyclictest30-21ksoftirqd/223:39:092
745855994533,9cyclictest30-21ksoftirqd/220:45:002
745855994530,9cyclictest30-21ksoftirqd/220:25:012
745855994528,13cyclictest1-21systemd21:10:012
745855994525,15cyclictest3467602-21systemd-journal20:40:012
745850994528,12cyclictest275-21dbus-daemon19:50:001
745850994525,14cyclictest275-21dbus-daemon19:15:011
745857994433,7cyclictest766211-21sh21:40:003
745857994430,9cyclictest775675-21sh22:00:003
745857994422,4cyclictest37-21ksoftirqd/322:42:403
745855994436,5cyclictest803878-21kworker/2:223:41:442
745855994436,5cyclictest508491-21kworker/2:221:26:322
745855994433,8cyclictest3467602-21systemd-journal19:12:282
745857994329,6cyclictest309-21in:imuxsock22:55:013
745857994328,7cyclictest309-21in:imuxsock21:00:003
745857994327,13cyclictest37-21ksoftirqd/300:34:133
745857994327,12cyclictest275-21dbus-daemon21:10:013
745857994326,12cyclictest1-21systemd00:10:013
745857994318,5cyclictest37-21ksoftirqd/322:24:263
745857994318,5cyclictest37-21ksoftirqd/322:24:263
74585799431,26cyclictest361rcuc/319:50:033
745855994326,11cyclictest30-21ksoftirqd/223:11:402
745855994324,14cyclictest275-21dbus-daemon19:30:002
74585099439,6cyclictest131rcu_preempt22:59:471
74585099432,16cyclictest843167-21ssh00:23:181
74585799424,3cyclictest131rcu_preempt23:37:453
745857994229,9cyclictest813263-21cron23:20:003
745857994229,10cyclictest37-21ksoftirqd/300:14:523
745857994228,11cyclictest37-21ksoftirqd/323:51:063
745857994221,14cyclictest789827-21modprobe22:30:003
745857994221,14cyclictest789827-21modprobe22:30:003
745857994218,18cyclictest275-21dbus-daemon00:30:003
745855994235,5cyclictest749949-21kworker/2:322:58:382
745855994234,5cyclictest749949-21kworker/2:320:50:022
745855994234,5cyclictest749949-21kworker/2:320:50:012
745855994234,5cyclictest508491-21kworker/2:222:08:442
745855994229,10cyclictest275-21dbus-daemon23:55:012
745855994222,14cyclictest275-21dbus-daemon19:20:012
745857994131,7cyclictest37-21ksoftirqd/300:03:003
745857994129,8cyclictest37-21ksoftirqd/321:50:003
745857994125,12cyclictest791106-21sshd22:32:353
745857994124,12cyclictest275-21dbus-daemon20:35:013
745857994116,8cyclictest131rcu_preempt21:29:063
745855994132,5cyclictest749949-21kworker/2:322:52:182
745855994127,10cyclictest309-21in:imuxsock20:57:292
745855994126,11cyclictest30-21ksoftirqd/220:55:002
745855994124,11cyclictest275-21dbus-daemon20:00:002
745850994112,9cyclictest0-21swapper/122:48:011
745850994110,6cyclictest131rcu_preempt20:55:001
74585799403,3cyclictest131rcu_preempt21:44:153
745857994026,11cyclictest37-21ksoftirqd/322:01:003
745857994025,12cyclictest37-21ksoftirqd/321:33:033
745857994020,15cyclictest275-21dbus-daemon19:20:013
745857994020,14cyclictest3467602-21systemd-journal20:00:023
745857994019,16cyclictest1-21systemd19:15:013
745850994032,5cyclictest778058-21latency_hist22:05:011
745857993930,5cyclictest806509-21kworker/u8:023:42:293
745857993925,11cyclictest37-21ksoftirqd/321:54:583
745857993925,11cyclictest37-21ksoftirqd/321:54:573
745857993925,10cyclictest850956-21systemctl00:40:003
745857993923,10cyclictest309-21in:imuxsock22:10:003
745857993922,11cyclictest309-21in:imuxsock19:30:003
745857993913,21cyclictest361rcuc/319:45:023
745855993930,6cyclictest508491-21kworker/2:219:22:502
745855993927,10cyclictest30-21ksoftirqd/222:42:402
745855993926,9cyclictest30-21ksoftirqd/200:27:122
74585599392,3cyclictest131rcu_preempt21:20:132
745857993828,6cyclictest807444-21sh23:07:233
745857993826,9cyclictest3467602-21systemd-journal20:37:283
745857993825,10cyclictest37-21ksoftirqd/323:20:163
745857993825,10cyclictest37-21ksoftirqd/323:20:153
745857993824,10cyclictest37-21ksoftirqd/323:50:003
74585799381,3cyclictest131rcu_preempt23:32:573
74585799381,3cyclictest131rcu_preempt23:32:573
745855993822,11cyclictest30-21ksoftirqd/219:40:002
745855993820,13cyclictest3467602-21systemd-journal20:10:002
745855993820,13cyclictest3467602-21systemd-journal20:10:002
74585799375,24cyclictest3467602-21systemd-journal19:55:003
745857993726,7cyclictest803798-21cron23:00:013
745855993711,4cyclictest131rcu_preempt20:04:422
74585099373,9cyclictest0-21swapper/122:25:561
74585099373,9cyclictest0-21swapper/122:25:561
74585099371,6cyclictest0-21swapper/123:24:291
74585099371,6cyclictest0-21swapper/123:24:291
745857993624,8cyclictest751728-21sshd21:01:303
745857993624,10cyclictest37-21ksoftirqd/323:26:333
745857993623,7cyclictest309-21in:imuxsock20:20:013
745857993623,10cyclictest37-21ksoftirqd/300:20:203
745857993622,11cyclictest37-21ksoftirqd/320:50:023
745857993622,11cyclictest37-21ksoftirqd/320:50:023
745857993612,6cyclictest37-21ksoftirqd/323:10:103
745855993627,6cyclictest30-21ksoftirqd/219:55:022
745850993628,4cyclictest748131-21latency_hist19:55:011
745850993623,10cyclictest1-21systemd20:35:011
745850993525,7cyclictest23-21ksoftirqd/119:29:281
74585799343,23cyclictest747380-21latency_hist19:40:013
745857993424,6cyclictest759262-21sh21:25:023
745855993421,10cyclictest30-21ksoftirqd/219:35:052
745855993421,10cyclictest30-21ksoftirqd/219:35:052
74585099349,4cyclictest131rcu_preempt20:15:001
745850993424,8cyclictest23-21ksoftirqd/120:10:021
745850993424,8cyclictest23-21ksoftirqd/120:10:021
745850993417,12cyclictest275-21dbus-daemon20:05:011
745857993319,10cyclictest748884-21cron20:10:003
745857993319,10cyclictest748884-21cron20:10:003
745850993324,6cyclictest747312-21sshd19:39:281
745850993321,7cyclictest751480-21sh21:00:001
745857993225,5cyclictest745774-21latency_hist19:10:013
745855993219,10cyclictest30-21ksoftirqd/220:14:592
74585799313,20cyclictest749928-21modprobe20:30:003
745857993119,9cyclictest746603-21sed19:25:023
745855993024,4cyclictest745774-21latency_hist19:10:022
74585099302,5cyclictest131rcu_preempt20:50:001
74585099302,5cyclictest131rcu_preempt20:50:001
74585099283,3cyclictest131rcu_preempt19:10:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional