You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-14 - 18:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu May 14, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
285643121210,3sleep30-21swapper/308:25:223
139099850,54rtkit-daemon0-21swapper/207:09:002
282813326444,14sleep00-21swapper/007:09:450
282680326241,14sleep30-21swapper/307:05:143
282780926140,14sleep10-21swapper/107:05:311
29167932600,3sleep32916798-21cpuspeed_turbos11:15:123
29238112540,3sleep10-21swapper/111:35:111
28940072510,3sleep10-21swapper/110:10:221
139099280,19rtkit-daemon0-21swapper/007:20:180
29270432270,3chrt0-21swapper/111:45:011
2828290992616,4cyclictest26-21ksoftirqd/107:23:311
28529632250,11sleep342-21ksoftirqd/308:15:183
282829999258,3cyclictest42-21ksoftirqd/310:09:553
282829999258,3cyclictest42-21ksoftirqd/310:09:543
282829399259,7cyclictest34-21ksoftirqd/211:00:182
282829399257,8cyclictest34-21ksoftirqd/210:40:012
282829099253,9cyclictest141rcu_preempt10:25:201
282828899251,9cyclictest2923554-21munin-run11:35:000
28658512240,8sleep234-21ksoftirqd/208:50:232
2828299992420,3cyclictest411rcuc/309:01:363
282829399244,6cyclictest2881981-21taskset09:38:512
282829399243,7cyclictest34-21ksoftirqd/209:46:112
28485972230,4sleep3411rcuc/308:05:013
2828299992315,6cyclictest2834196-21cron07:24:593
2828299992314,7cyclictest2870541-21memory09:05:173
282829399234,7cyclictest34-21ksoftirqd/211:19:472
282829399233,6cyclictest34-21ksoftirqd/210:40:212
282829399233,3cyclictest34-21ksoftirqd/207:25:112
2828293992316,5cyclictest2883807-21cron09:45:012
282829099239,4cyclictest26-21ksoftirqd/112:20:161
28763172220,12sleep00-21swapper/009:21:140
2828299992220,1cyclictest0-21swapper/307:55:303
2828299992219,2cyclictest3056-21PK-Backend12:33:393
2828299992219,2cyclictest3056-21PK-Backend11:31:193
2828299992218,3cyclictest411rcuc/311:06:213
2828293992220,1cyclictest0-21swapper/212:20:082
2828293992220,1cyclictest0-21swapper/211:59:062
2828293992220,1cyclictest0-21swapper/208:43:172
2828293992218,3cyclictest331rcuc/210:09:282
2828293992218,3cyclictest331rcuc/210:09:282
282829099227,2cyclictest141rcu_preempt11:35:001
2828290992211,10cyclictest26-21ksoftirqd/111:10:011
2828290992210,4cyclictest26-21ksoftirqd/110:05:011
2828290992210,4cyclictest26-21ksoftirqd/110:05:001
2828288992220,1cyclictest3056-21PK-Backend11:38:130
2828288992220,1cyclictest0-21swapper/010:58:400
2828288992220,1cyclictest0-21swapper/010:53:530
2828288992220,1cyclictest0-21swapper/010:39:320
2828288992220,1cyclictest0-21swapper/010:32:430
2828288992220,1cyclictest0-21swapper/009:56:070
2828288992220,1cyclictest0-21swapper/009:11:590
2828288992220,1cyclictest0-21swapper/008:06:570
2828288992220,1cyclictest0-21swapper/007:31:010
2828288992219,2cyclictest3056-21PK-Backend08:54:080
282828899220,21cyclictest3056-21PK-Backend10:44:190
282828899220,21cyclictest0-21swapper/009:41:400
282828899220,21cyclictest0-21swapper/007:50:140
282828899220,20cyclictest0-21swapper/009:46:470
139099220,15rtkit-daemon3056-21PK-Backend11:23:201
139099220,14rtkit-daemon0-21swapper/111:26:361
29023582210,2sleep13056-21PK-Backend10:35:161
28504652210,8sleep0131ksoftirqd/008:10:100
282829999212,3cyclictest42-21ksoftirqd/310:10:193
2828299992119,1cyclictest3056-21PK-Backend10:22:593
2828299992119,1cyclictest0-21swapper/311:52:013
2828299992119,1cyclictest0-21swapper/310:41:103
2828299992118,2cyclictest411rcuc/310:17:013
2828299992118,2cyclictest3056-21PK-Backend10:28:333
2828299992117,3cyclictest411rcuc/307:55:003
2828299992117,3cyclictest411rcuc/307:25:033
282829999211,17cyclictest2878021-21python309:25:233
282829999210,20cyclictest3056-21PK-Backend10:55:303
282829999210,20cyclictest0-21swapper/312:23:443
282829999210,20cyclictest0-21swapper/311:20:593
282829399218,3cyclictest34-21ksoftirqd/211:05:282
2828293992119,1cyclictest0-21swapper/212:27:442
2828293992119,1cyclictest0-21swapper/211:53:312
2828293992119,1cyclictest0-21swapper/210:59:382
2828293992119,1cyclictest0-21swapper/210:48:492
2828293992119,1cyclictest0-21swapper/210:32:132
2828293992119,1cyclictest0-21swapper/210:25:242
2828293992119,1cyclictest0-21swapper/210:16:032
2828293992119,1cyclictest0-21swapper/210:14:492
2828293992119,1cyclictest0-21swapper/209:57:112
2828293992119,1cyclictest0-21swapper/209:52:242
2828293992119,1cyclictest0-21swapper/209:26:422
2828293992119,1cyclictest0-21swapper/209:06:022
2828293992119,1cyclictest0-21swapper/208:00:312
2828293992119,1cyclictest0-21swapper/207:43:012
2828293992119,1cyclictest0-21swapper/207:38:012
282829399211,7cyclictest2896861-21diskstats10:20:142
2828293992117,3cyclictest0-21swapper/209:04:592
2828293992112,7cyclictest883-21dbus-daemon10:53:322
282829399210,20cyclictest0-21swapper/212:37:382
282829399210,20cyclictest0-21swapper/212:04:402
282829399210,20cyclictest0-21swapper/212:04:392
282829099219,4cyclictest26-21ksoftirqd/108:15:291
282829099217,11cyclictest883-21dbus-daemon10:53:351
2828290992119,1cyclictest0-21swapper/110:01:351
282829099210,20cyclictest3056-21PK-Backend11:47:001
282829099210,19cyclictest3056-21PK-Backend09:00:451
282829099210,19cyclictest3056-21PK-Backend08:00:031
282829099210,19cyclictest3056-21PK-Backend07:52:251
2828288992119,1cyclictest3056-21PK-Backend12:18:490
2828288992119,1cyclictest0-21swapper/012:37:080
2828288992119,1cyclictest0-21swapper/012:32:490
2828288992119,1cyclictest0-21swapper/012:02:470
2828288992119,1cyclictest0-21swapper/012:02:470
2828288992119,1cyclictest0-21swapper/009:39:370
2828288992119,1cyclictest0-21swapper/009:09:570
2828288992119,1cyclictest0-21swapper/009:01:390
2828288992119,1cyclictest0-21swapper/008:33:060
2828288992119,1cyclictest0-21swapper/007:47:360
2828288992119,1cyclictest0-21swapper/007:44:590
282828899210,20cyclictest3056-21PK-Backend11:59:290
282828899210,20cyclictest3056-21PK-Backend08:35:430
282828899210,20cyclictest0-21swapper/011:28:540
282828899210,20cyclictest0-21swapper/010:23:380
282828899210,19cyclictest2894512-21systemd-run10:15:000
139099210,15rtkit-daemon3056-21PK-Backend09:30:591
139099210,14rtkit-daemon3056-21PK-Backend09:46:211
139099210,14rtkit-daemon3056-21PK-Backend09:22:351
139099210,14rtkit-daemon3056-21PK-Backend08:55:091
139099210,14rtkit-daemon3056-21PK-Backend08:35:091
2828299992018,1cyclictest3056-21PK-Backend11:01:173
2828299992018,1cyclictest0-21swapper/311:49:453
2828299992018,1cyclictest0-21swapper/309:45:203
2828299992018,1cyclictest0-21swapper/309:13:563
2828299992018,1cyclictest0-21swapper/308:50:583
2828299992018,1cyclictest0-21swapper/308:31:323
2828299992018,1cyclictest0-21swapper/308:22:533
2828299992018,1cyclictest0-21swapper/307:41:373
2828299992018,1cyclictest0-21swapper/307:15:133
2828299992017,2cyclictest411rcuc/312:18:533
2828299992017,2cyclictest411rcuc/307:31:143
2828299992016,3cyclictest411rcuc/311:44:393
2828299992016,3cyclictest411rcuc/309:18:243
2828299992016,3cyclictest411rcuc/308:37:553
2828299992013,5cyclictest2863659-21cat08:45:163
2828299992012,6cyclictest2915355-21ntp_states11:10:163
2828299992012,6cyclictest2902679-21ntp_kernel_pll_10:35:183
2828299992012,6cyclictest2883429-21users09:40:223
2828299992012,6cyclictest2829957-21apt-get07:15:003
282829999200,8cyclictest2879261-21taskset09:30:133
282829999200,19cyclictest3056-21PK-Backend12:40:013
282829999200,19cyclictest3056-21PK-Backend11:35:513
282829999200,19cyclictest3056-21PK-Backend11:26:473
282829999200,19cyclictest0-21swapper/310:31:513
282829999200,19cyclictest0-21swapper/309:22:213
282829999200,18cyclictest2850369-21apt08:10:103
282829399202,2cyclictest141rcu_preempt07:20:192
2828293992018,1cyclictest0-21swapper/212:16:502
2828293992018,1cyclictest0-21swapper/212:13:322
2828293992018,1cyclictest0-21swapper/209:32:522
2828293992018,1cyclictest0-21swapper/209:19:202
2828293992018,1cyclictest0-21swapper/209:13:592
2828293992018,1cyclictest0-21swapper/208:28:432
2828293992018,1cyclictest0-21swapper/208:09:242
2828293992018,1cyclictest0-21swapper/207:58:412
2828293992017,2cyclictest0-21swapper/212:05:542
2828293992017,2cyclictest0-21swapper/211:25:472
2828293992017,2cyclictest0-21swapper/207:11:512
2828293992016,3cyclictest331rcuc/210:04:452
2828293992014,5cyclictest883-21dbus-daemon11:38:032
282829399200,1cyclictest0-21swapper/208:34:442
282829399200,19cyclictest0-21swapper/208:16:402
282829399200,19cyclictest0-21swapper/208:11:132
282829099209,4cyclictest26-21ksoftirqd/110:00:001
282829099209,3cyclictest26-21ksoftirqd/107:10:081
282829099209,3cyclictest141rcu_preempt07:25:121
2828290992018,1cyclictest0-21swapper/112:15:191
2828290992017,2cyclictest3056-21PK-Backend07:35:141
2828290992015,3cyclictest2881484-21python309:35:191
2828290992012,6cyclictest2911144-21expr11:00:121
2828290992011,7cyclictest2896488-21apt10:20:121
2828290992010,4cyclictest26-21ksoftirqd/110:30:221
2828290992010,4cyclictest26-21ksoftirqd/110:15:221
2828290992010,4cyclictest26-21ksoftirqd/107:59:171
282829099200,19cyclictest3056-21PK-Backend08:41:591
282829099200,19cyclictest3056-21PK-Backend08:22:201
282829099200,18cyclictest3056-21PK-Backend09:15:331
282829099200,18cyclictest3056-21PK-Backend08:32:191
282829099200,18cyclictest3056-21PK-Backend07:30:281
2828288992018,1cyclictest0-21swapper/012:24:100
2828288992018,1cyclictest0-21swapper/012:05:040
2828288992018,1cyclictest0-21swapper/011:46:180
2828288992018,1cyclictest0-21swapper/011:42:470
2828288992018,1cyclictest0-21swapper/011:22:050
2828288992018,1cyclictest0-21swapper/008:55:100
2828288992018,1cyclictest0-21swapper/007:55:280
2828288992016,2cyclictest2848366-21sh08:05:000
2828288992014,4cyclictest2942881-21perf12:30:000
282828899200,1cyclictest3056-21PK-Backend07:12:150
282828899200,1cyclictest0-21swapper/012:11:260
282828899200,19cyclictest0-21swapper/011:51:530
282828899200,19cyclictest0-21swapper/010:02:560
282828899200,19cyclictest0-21swapper/009:50:190
282828899200,19cyclictest0-21swapper/008:27:050
139099200,14rtkit-daemon0-21swapper/008:44:140
139099200,13rtkit-daemon0-21swapper/110:44:041
2828299991917,1cyclictest3056-21PK-Backend09:59:133
2828299991917,1cyclictest0-21swapper/311:58:363
2828299991917,1cyclictest0-21swapper/308:58:293
2828299991915,3cyclictest411rcuc/312:12:113
2828299991914,3cyclictest2886193-21diskstats09:50:113
2828299991912,6cyclictest883-21dbus-daemon12:10:013
2828299991912,5cyclictest436-20systemd-journal12:02:533
2828299991912,5cyclictest436-20systemd-journal12:02:533
2828299991911,6cyclictest2908141-21ntpq10:50:183
282829999190,1cyclictest3056-21PK-Backend10:03:003
282829999190,1cyclictest0-21swapper/308:44:173
282829999190,1cyclictest0-21swapper/308:09:363
282829999190,18cyclictest0-21swapper/310:47:113
282829999190,18cyclictest0-21swapper/307:45:013
282829999190,18cyclictest0-21swapper/307:35:493
2828293991917,1cyclictest0-21swapper/207:54:162
2828293991916,2cyclictest0-21swapper/211:42:082
2828293991913,5cyclictest34-21ksoftirqd/211:20:102
2828293991912,6cyclictest2841499-21awk07:45:102
2828293991911,6cyclictest2830362-21systemd07:15:012
282829399190,1cyclictest0-21swapper/211:46:412
282829399190,1cyclictest0-21swapper/211:31:352
282829399190,1cyclictest0-21swapper/208:56:272
282829399190,18cyclictest0-21swapper/212:31:022
282829399190,18cyclictest0-21swapper/211:12:412
282829399190,18cyclictest0-21swapper/208:46:082
282829099199,4cyclictest26-21ksoftirqd/107:46:101
282829099198,2cyclictest141rcu_preempt10:55:231
282829099197,4cyclictest141rcu_preempt07:15:041
2828290991916,2cyclictest3056-21PK-Backend09:09:371
282829099190,7cyclictest2944604-21cron12:35:011
282829099190,6cyclictest2930486-21awk11:54:591
282829099190,2cyclictest3056-21PK-Backend07:45:011
282829099190,1cyclictest3056-21PK-Backend12:05:541
282829099190,1cyclictest3056-21PK-Backend09:12:281
282829099190,1cyclictest3056-21PK-Backend08:06:511
282829099190,18cyclictest3056-21PK-Backend08:29:431
282829099190,18cyclictest3056-21PK-Backend08:12:051
282828899195,5cyclictest13-21ksoftirqd/010:50:010
2828288991917,1cyclictest0-21swapper/010:15:470
2828288991917,1cyclictest0-21swapper/008:23:390
282828899190,1cyclictest0-21swapper/011:11:300
282828899190,1cyclictest0-21swapper/009:34:310
139099190,13rtkit-daemon0-21swapper/007:15:080
2828299991811,5cyclictest2941404-21apt12:25:093
2828299991810,6cyclictest436-20systemd-journal09:36:543
2828293991816,1cyclictest0-21swapper/208:35:472
2828293991816,1cyclictest0-21swapper/208:21:542
2828293991810,6cyclictest2875529-21df09:20:122
282829099188,3cyclictest141rcu_preempt12:36:081
282829099187,4cyclictest26-21ksoftirqd/112:00:491
282829099187,4cyclictest26-21ksoftirqd/112:00:491
2828290991816,1cyclictest3056-21PK-Backend09:54:471
2828290991816,1cyclictest3056-21PK-Backend09:28:231
282829099180,6cyclictest3056-21PK-Backend11:55:221
282829099180,5cyclictest3056-21PK-Backend12:25:101
282829099180,5cyclictest3056-21PK-Backend12:10:081
282829099180,1cyclictest3056-21PK-Backend08:50:521
282829099180,1cyclictest0-21swapper/110:45:371
2828288991816,1cyclictest0-21swapper/011:18:340
2828288991816,1cyclictest0-21swapper/011:09:130
2828288991816,1cyclictest0-21swapper/010:08:440
2828288991816,1cyclictest0-21swapper/010:08:430
2828288991816,1cyclictest0-21swapper/008:16:500
2828288991811,5cyclictest2911945-21cat11:00:210
2828288991811,5cyclictest2864615-21grep08:50:000
282829399170,7cyclictest436-20systemd-journal07:30:002
282829099178,3cyclictest26-21ksoftirqd/111:10:171
282829099178,3cyclictest26-21ksoftirqd/109:42:511
2828290991714,2cyclictest3056-21PK-Backend11:19:571
2828290991710,5cyclictest26-21ksoftirqd/108:47:361
2828288991715,1cyclictest0-21swapper/010:27:100
2828288991715,1cyclictest0-21swapper/007:25:270
2828288991614,1cyclictest0-21swapper/009:25:320
2828288991611,4cyclictest2839460-21basename07:40:010
2828288991511,3cyclictest3056-21PK-Backend09:19:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional