You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 03:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Tue Aug 11, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
256381421170,6sleep1261ksoftirqd/120:35:211
261742921110,9sleep234-21ksoftirqd/223:10:162
253209626341,15sleep20-21swapper/219:09:222
253193426343,13sleep10-21swapper/119:07:151
253208726243,13sleep30-21swapper/319:09:143
253192126242,13sleep00-21swapper/019:07:060
25621352510,3sleep20-21swapper/220:30:202
26054192460,2sleep20-21swapper/222:35:152
139099320,4rtkit-daemon1389-21rtkit-daemon21:25:442
139099320,4rtkit-daemon1389-21rtkit-daemon20:25:542
139099310,4rtkit-daemon1389-21rtkit-daemon23:15:392
139099300,4rtkit-daemon1389-21rtkit-daemon20:10:192
139099280,4rtkit-daemon1389-21rtkit-daemon22:34:412
139099280,3rtkit-daemon1389-21rtkit-daemon22:09:482
139099280,3rtkit-daemon1389-21rtkit-daemon22:09:482
139099270,4rtkit-daemon1389-21rtkit-daemon21:49:543
26341342250,3sleep23056-21PK-Backend00:00:002
2532367992411,10cyclictest883-21dbus-daemon23:25:001
139099240,4rtkit-daemon1389-21rtkit-daemon19:51:543
2532378992315,6cyclictest2542648-21ntp_states19:35:173
2532378992315,6cyclictest2176-21JS10:27:483
2532378992315,6cyclictest2176-21JS10:27:483
2532378992314,6cyclictest436-20systemd-journal19:20:003
253237399230,21cyclictest1389-21rtkit-daemon20:35:592
2532367992320,2cyclictest3056-21PK-Backend19:32:571
2532367992318,3cyclictest2632111-21chrt23:52:491
2532367992317,5cyclictest3056-21PK-Backend21:17:071
139099230,4rtkit-daemon1020-21ntpd21:06:193
25717792220,3sleep22571784-21cpuspeed_turbos21:00:132
25717792220,3sleep22571784-21cpuspeed_turbos21:00:122
2532378992220,1cyclictest0-21swapper/321:27:303
2532378992220,1cyclictest0-21swapper/321:23:173
2532378992220,1cyclictest0-21swapper/321:02:083
2532378992220,1cyclictest0-21swapper/321:02:083
2532378992220,1cyclictest0-21swapper/320:42:423
253237899220,20cyclictest3056-21PK-Backend23:47:013
2532373992214,6cyclictest2578282-21perf21:20:002
253237399220,2cyclictest2156-21gnome-shell23:26:092
2532367992220,1cyclictest3056-21PK-Backend21:11:261
2532367992220,1cyclictest3056-21PK-Backend20:15:491
2532367992219,2cyclictest3056-21PK-Backend21:32:291
2532367992217,3cyclictest2645160-21expr00:30:121
253236799220,20cyclictest3056-21PK-Backend23:10:021
253236799220,20cyclictest3056-21PK-Backend22:59:411
2532363992215,5cyclictest2634018-21cron23:59:590
139099220,4rtkit-daemon1389-21rtkit-daemon00:20:293
2532378992119,1cyclictest3056-21PK-Backend00:27:173
2532378992119,1cyclictest0-21swapper/323:30:063
2532378992119,1cyclictest0-21swapper/323:16:463
2532378992119,1cyclictest0-21swapper/322:51:453
2532378992119,1cyclictest0-21swapper/322:47:333
2532378992119,1cyclictest0-21swapper/322:12:393
2532378992119,1cyclictest0-21swapper/321:18:323
2532378992119,1cyclictest0-21swapper/320:56:273
2532378992119,1cyclictest0-21swapper/320:36:533
2532378992119,1cyclictest0-21swapper/320:33:083
2532378992119,1cyclictest0-21swapper/320:10:443
2532378992119,1cyclictest0-21swapper/319:58:563
2532378992119,1cyclictest0-21swapper/300:33:323
2532378992118,2cyclictest3056-21PK-Backend23:37:013
2532378992118,2cyclictest3056-21PK-Backend19:42:463
253237899211,18cyclictest2575829-21ntp_states21:10:203
253237899210,20cyclictest0-21swapper/322:38:353
253237899210,19cyclictest3056-21PK-Backend00:16:233
2532373992119,1cyclictest0-21swapper/200:27:482
2532373992115,4cyclictest2156-21gnome-shell23:04:092
2532373992115,4cyclictest2156-21gnome-shell23:04:092
2532373992111,8cyclictest2584071-21cstates21:35:132
253237399210,20cyclictest0-21swapper/221:49:162
253237399210,20cyclictest0-21swapper/200:16:382
2532367992119,1cyclictest3056-21PK-Backend22:49:341
2532367992119,1cyclictest3056-21PK-Backend19:46:561
2532367992119,1cyclictest3056-21PK-Backend19:14:121
2532367992119,1cyclictest0-21swapper/123:47:011
2532367992119,1cyclictest0-21swapper/120:23:541
2532367992119,1cyclictest0-21swapper/119:42:451
2532367992118,2cyclictest3056-21PK-Backend21:48:511
2532367992118,2cyclictest3056-21PK-Backend21:36:141
2532367992118,2cyclictest3056-21PK-Backend20:51:251
253236799211,17cyclictest2599362-21idleruntime-cro22:20:001
253236799210,20cyclictest3056-21PK-Backend19:57:171
253236799210,19cyclictest3056-21PK-Backend22:05:411
253236799210,19cyclictest3056-21PK-Backend22:05:401
253236799210,19cyclictest3056-21PK-Backend21:27:351
253236799210,19cyclictest3056-21PK-Backend20:47:401
253236799210,19cyclictest3056-21PK-Backend20:10:351
253236799210,19cyclictest3056-21PK-Backend00:37:031
2532363992114,5cyclictest2644304-21uptime00:25:210
253236399210,20cyclictest0-21swapper/023:26:370
253236399210,20cyclictest0-21swapper/022:35:340
253236399210,20cyclictest0-21swapper/021:24:080
253236399210,20cyclictest0-21swapper/021:12:190
139099210,3rtkit-daemon1389-21rtkit-daemon21:35:233
2532378992018,1cyclictest3056-21PK-Backend23:20:383
2532378992018,1cyclictest0-21swapper/322:42:463
2532378992018,1cyclictest0-21swapper/322:32:053
2532378992018,1cyclictest0-21swapper/320:53:373
2532378992018,1cyclictest0-21swapper/320:47:023
2532378992018,1cyclictest0-21swapper/320:17:213
2532378992018,1cyclictest0-21swapper/320:09:223
2532378992018,1cyclictest0-21swapper/319:46:243
2532378992017,2cyclictest3056-21PK-Backend00:11:083
2532378992017,2cyclictest3056-21PK-Backend00:06:283
253237899200,19cyclictest0-21swapper/323:10:243
253237899200,19cyclictest0-21swapper/323:05:243
253237899200,19cyclictest0-21swapper/322:55:583
253237899200,19cyclictest0-21swapper/322:05:013
253237899200,19cyclictest0-21swapper/322:05:013
253237899200,19cyclictest0-21swapper/321:55:423
253237899200,19cyclictest0-21swapper/320:25:243
253237899200,19cyclictest0-21swapper/320:21:063
253237899200,18cyclictest3056-21PK-Backend23:28:103
2532373992018,1cyclictest0-21swapper/223:47:592
2532373992018,1cyclictest0-21swapper/223:44:412
2532373992018,1cyclictest0-21swapper/222:53:102
2532373992018,1cyclictest0-21swapper/222:03:422
2532373992018,1cyclictest0-21swapper/220:24:132
2532373992018,1cyclictest0-21swapper/220:03:522
2532373992018,1cyclictest0-21swapper/219:56:082
2532373992018,1cyclictest0-21swapper/219:26:532
2532373992018,1cyclictest0-21swapper/200:35:382
253237399200,19cyclictest0-21swapper/222:58:442
253237399200,19cyclictest0-21swapper/221:41:182
253237399200,19cyclictest0-21swapper/220:50:562
253237399200,19cyclictest0-21swapper/220:08:302
253237399200,19cyclictest0-21swapper/219:52:152
253237399200,19cyclictest0-21swapper/219:20:172
253237399200,19cyclictest0-21swapper/219:19:432
2532367992018,1cyclictest3056-21PK-Backend22:34:331
2532367992018,1cyclictest3056-21PK-Backend21:50:331
2532367992018,1cyclictest3056-21PK-Backend20:06:101
2532367992018,1cyclictest3056-21PK-Backend20:01:431
2532367992018,1cyclictest3056-21PK-Backend19:54:471
2532367992018,1cyclictest3056-21PK-Backend19:22:431
2532367992018,1cyclictest0-21swapper/100:12:281
2532367992017,2cyclictest3056-21PK-Backend21:09:091
2532367992011,6cyclictest2640965-21cron00:20:001
253236799200,1cyclictest3056-21PK-Backend21:59:251
253236799200,1cyclictest0-21swapper/100:20:261
253236799200,19cyclictest0-21swapper/123:25:371
253236799200,19cyclictest0-21swapper/100:25:191
253236799200,19cyclictest0-21swapper/100:03:081
253236799200,19cyclictest0-21swapper/100:03:081
253236799200,18cyclictest3056-21PK-Backend22:02:171
253236799200,18cyclictest3056-21PK-Backend21:02:551
253236799200,18cyclictest3056-21PK-Backend21:02:541
2532363992018,1cyclictest0-21swapper/023:51:320
2532363992018,1cyclictest0-21swapper/022:46:480
2532363992018,1cyclictest0-21swapper/020:49:000
2532363992018,1cyclictest0-21swapper/020:01:350
2532363992018,1cyclictest0-21swapper/019:55:130
2532363992018,1cyclictest0-21swapper/019:21:000
2532363992018,1cyclictest0-21swapper/000:35:310
253236399200,1cyclictest0-21swapper/023:08:520
253236399200,1cyclictest0-21swapper/022:42:020
253236399200,1cyclictest0-21swapper/020:59:000
253236399200,1cyclictest0-21swapper/020:29:060
253236399200,19cyclictest0-21swapper/023:35:290
253236399200,19cyclictest0-21swapper/023:22:110
253236399200,19cyclictest0-21swapper/022:58:530
253236399200,19cyclictest0-21swapper/021:34:220
253236399200,19cyclictest0-21swapper/021:19:220
253236399200,19cyclictest0-21swapper/020:35:420
253236399200,19cyclictest0-21swapper/020:21:010
253236399200,19cyclictest0-21swapper/019:31:470
253236399200,19cyclictest0-21swapper/019:28:090
253236399200,19cyclictest0-21swapper/000:33:220
253236399200,19cyclictest0-21swapper/000:06:520
2532378991917,1cyclictest0-21swapper/322:26:443
2532378991917,1cyclictest0-21swapper/321:40:563
2532378991917,1cyclictest0-21swapper/319:27:263
2532378991914,4cyclictest2583450-21fwupdmgr21:34:443
2532378991913,5cyclictest2533327-21memory19:10:173
2532378991912,6cyclictest2593192-21awk22:00:203
2532378991911,6cyclictest2589516-21cat21:50:173
253237899190,1cyclictest3056-21PK-Backend23:55:483
253237899190,1cyclictest0-21swapper/320:02:463
253237899190,1cyclictest0-21swapper/319:22:193
2532373991917,1cyclictest0-21swapper/223:34:272
2532373991917,1cyclictest0-21swapper/220:15:202
2532373991917,1cyclictest0-21swapper/219:47:142
2532373991917,1cyclictest0-21swapper/219:30:332
2532373991917,1cyclictest0-21swapper/200:01:042
2532373991917,1cyclictest0-21swapper/200:01:042
2532373991913,4cyclictest978-21snmpd22:40:092
2532373991912,6cyclictest2597384-21systemd-run22:15:002
2532373991912,6cyclictest2571152-21seq20:57:382
2532373991912,5cyclictest34-21ksoftirqd/219:35:102
253237399190,1cyclictest0-21swapper/223:07:382
253237399190,1cyclictest0-21swapper/220:49:132
253237399190,1cyclictest0-21swapper/220:40:342
253237399190,1cyclictest0-21swapper/200:09:492
253237399190,18cyclictest0-21swapper/223:36:242
253237399190,18cyclictest0-21swapper/221:33:482
253237399190,18cyclictest0-21swapper/200:34:502
2532367991917,1cyclictest3056-21PK-Backend23:15:371
2532367991917,1cyclictest3056-21PK-Backend23:02:451
2532367991917,1cyclictest3056-21PK-Backend23:02:451
2532367991917,1cyclictest0-21swapper/123:42:331
2532367991916,2cyclictest3056-21PK-Backend23:35:031
2532367991913,4cyclictest2571355-21idleruntime-cro20:59:591
2532367991912,6cyclictest2600261-21needreboot22:20:181
253236799190,18cyclictest3056-21PK-Backend19:18:381
2532363991917,1cyclictest0-21swapper/023:10:480
2532363991917,1cyclictest0-21swapper/022:20:530
2532363991917,1cyclictest0-21swapper/020:41:300
2532363991917,1cyclictest0-21swapper/019:40:260
2532363991917,1cyclictest0-21swapper/000:23:350
2532363991917,1cyclictest0-21swapper/000:16:110
2532363991914,3cyclictest1095-21runrttasks23:40:130
253236399190,1cyclictest0-21swapper/023:47:250
253236399190,1cyclictest0-21swapper/021:35:300
253236399190,1cyclictest0-21swapper/019:50:470
253236399190,1cyclictest0-21swapper/019:48:170
253236399190,1cyclictest0-21swapper/019:38:300
253236399190,1cyclictest0-21swapper/019:18:220
253236399190,18cyclictest0-21swapper/021:27:460
253236399190,18cyclictest0-21swapper/020:12:030
26343022180,4chrt3056-21PK-Backend00:00:133
26343022180,4chrt3056-21PK-Backend00:00:133
253237899186,11cyclictest42-21ksoftirqd/322:25:003
2532378991816,1cyclictest3056-21PK-Backend22:19:413
2532378991816,1cyclictest0-21swapper/300:37:033
2532378991812,5cyclictest2540429-21expr19:30:143
253237399188,9cyclictest2534243-21unin-run19:15:002
2532367991816,1cyclictest3056-21PK-Backend21:41:411
2532367991816,1cyclictest3056-21PK-Backend20:31:591
2532367991816,1cyclictest0-21swapper/120:25:021
2532367991816,1cyclictest0-21swapper/100:05:531
2532363991816,1cyclictest0-21swapper/023:15:140
2532363991816,1cyclictest0-21swapper/022:51:350
2532363991816,1cyclictest0-21swapper/022:29:380
2532363991816,1cyclictest0-21swapper/020:08:440
2532363991816,1cyclictest0-21swapper/019:13:220
2532363991816,1cyclictest0-21swapper/000:14:500
2532363991812,4cyclictest2585936-21awk21:40:170
2532363991810,6cyclictest2598584-21ntp_kernel_err22:15:180
253236399180,1cyclictest0-21swapper/020:17:170
253237899172,13cyclictest2627343-21/usr/sbin/munin23:40:183
2532378991715,1cyclictest3056-21PK-Backend23:51:073
253237399179,6cyclictest2543815-21expr19:40:132
2532373991715,1cyclictest0-21swapper/200:22:062
2532373991715,1cyclictest0-21swapper/200:10:362
2532373991714,2cyclictest2632286-21sh23:54:592
2532373991711,5cyclictest2576671-21ls21:15:002
2532373991710,5cyclictest2608944-21grep22:45:202
2532367991715,1cyclictest3056-21PK-Backend20:42:261
2532367991714,2cyclictest3056-21PK-Backend23:05:501
2532367991711,5cyclictest2633006-21cat23:55:141
253236399179,2cyclictest13-21ksoftirqd/021:59:550
2532363991715,1cyclictest0-21swapper/022:14:250
2532363991715,1cyclictest0-21swapper/020:31:220
2532363991714,2cyclictest0-21swapper/000:00:160
2532363991714,2cyclictest0-21swapper/000:00:160
2532363991712,3cyclictest2594826-21needreboot22:05:190
2532363991712,3cyclictest2594826-21needreboot22:05:180
253237399169,5cyclictest2598930-21cat22:15:232
253237399169,5cyclictest2590478-21date21:55:022
253237399167,7cyclictest436-20systemd-journal22:25:012
2532373991614,1cyclictest0-21swapper/223:21:432
2532373991614,1cyclictest0-21swapper/221:23:412
2532373991611,4cyclictest2602243-21sed22:25:212
2532367991614,1cyclictest3056-21PK-Backend19:36:021
2532367991612,3cyclictest2624112-21cstates23:30:131
253236799161,13cyclictest2597357-21grep22:15:001
253236799161,11cyclictest2539907-21turbostat19:30:001
253236399169,5cyclictest2614055-21cat23:00:170
253236399169,5cyclictest2614055-21cat23:00:160
253236399168,6cyclictest883-21dbus-daemon22:34:110
2532363991614,1cyclictest0-21swapper/020:50:420
253237399159,4cyclictest2591593-21switchtime21:55:212
253236799153,4cyclictest26-21ksoftirqd/122:50:391
253236399159,4cyclictest2589633-21cut21:50:180
253236399158,5cyclictest2624913-21grep23:30:220
2532363991511,3cyclictest2571597-21/usr/sbin/munin21:00:160
2532363991511,3cyclictest2571597-21/usr/sbin/munin21:00:160
253237399149,3cyclictest2573331-21/usr/sbin/munin21:05:122
253236799144,2cyclictest141rcu_preempt22:25:211
2532363991410,2cyclictest2593882-21latency_hist22:05:000
26073092130,4sleep13056-21PK-Backend22:40:191
253236799139,3cyclictest3056-21PK-Backend22:37:551
253236799139,3cyclictest3056-21PK-Backend21:21:041
253236399139,3cyclictest2587200-21/usr/sbin/munin21:45:160
253236399122,9cyclictest2573948-21missed_timers21:05:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional