You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 14:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Mon Jun 29, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
342420621030,8sleep3411rcuc/319:05:143
1390991000,57rtkit-daemon0-21swapper/219:06:382
342469227645,22sleep10-21swapper/119:05:251
342492726844,15sleep00-21swapper/019:05:280
34284282480,2sleep10-21swapper/119:15:161
35285172470,4chrt3056-21PK-Backend00:00:221
34855242450,3sleep126-21ksoftirqd/122:00:001
34855242450,3sleep126-21ksoftirqd/122:00:001
139099290,4rtkit-daemon1389-21rtkit-daemon22:51:480
34485932280,2sleep3411rcuc/320:15:003
342551099289,5cyclictest918-21rs:main2
3425510992711,5cyclictest3449929-21cron20:17:002
3425510992612,3cyclictest978-21snmpd23:47:012
3425510992612,3cyclictest3485826-21cpuspeed_turbos22:00:142
139099260,4rtkit-daemon1389-21rtkit-daemon23:28:481
139099260,4rtkit-daemon1389-21rtkit-daemon00:10:511
34609682250,3sleep33056-21PK-Backend20:50:013
3425510992520,4cyclictest0-21swapper/223:19:002
139099250,4rtkit-daemon1389-21rtkit-daemon22:26:041
342551499240,22cyclictest3474966-21runrttasks21:29:493
3425510992419,4cyclictest0-21swapper/223:55:032
3425510992419,3cyclictest3475847-21memory21:30:182
3425510992418,5cyclictest0-21swapper/221:28:422
342550599241,8cyclictest34415422awk19:55:001
3425505992415,7cyclictest3492316-21cron22:20:001
139099240,3rtkit-daemon1389-21rtkit-daemon23:12:361
35326052230,3chrt3056-21PK-Backend00:15:010
3425510992321,1cyclictest0-21swapper/219:20:332
3425510992318,4cyclictest0-21swapper/220:53:132
3425510992317,4cyclictest3446960-21/usr/sbin/munin20:10:142
342551099230,21cyclictest3469480-21taskset21:11:182
3425505992317,4cyclictest3530757-21idleruntime-cro00:10:001
3425505992314,7cyclictest3506324-21sshd22:58:251
3425505992314,7cyclictest3506324-21sshd22:58:241
342550599230,21cyclictest3509572-21seq23:05:401
139099230,3rtkit-daemon1389-21rtkit-daemon22:40:531
139099230,15rtkit-daemon3056-21PK-Backend20:40:353
35014862220,4sleep00-21swapper/022:45:020
34831782220,3chrt3483181-21sort21:50:210
342551499225,15cyclictest3504730-21sh22:55:003
3425514992217,3cyclictest3472919-21taskset21:20:563
342551499220,20cyclictest3056-21PK-Backend19:50:113
342551499220,20cyclictest3056-21PK-Backend19:33:423
342551099228,12cyclictest3508225-21unin-run23:05:002
3425510992220,1cyclictest0-21swapper/223:54:292
3425510992220,1cyclictest0-21swapper/221:46:132
3425510992220,1cyclictest0-21swapper/219:50:002
3425510992220,1cyclictest0-21swapper/200:16:062
3425510992214,6cyclictest3527922-21df_inode00:00:132
342551099221,20cyclictest34-21ksoftirqd/223:44:152
3425505992220,1cyclictest0-21swapper/123:19:121
3425505992219,2cyclictest3056-21PK-Backend20:15:261
3425505992214,6cyclictest1-21systemd21:10:001
3425505992210,9cyclictest883-21dbus-daemon20:35:481
342550599220,21cyclictest3056-21PK-Backend23:57:171
342550599220,21cyclictest1389-21rtkit-daemon22:05:221
342551499210,19cyclictest3470791-21ntp_states21:15:193
342551499210,19cyclictest3056-21PK-Backend22:27:543
342551499210,19cyclictest3056-21PK-Backend21:10:073
3425510992119,1cyclictest0-21swapper/222:43:112
3425510992119,1cyclictest0-21swapper/222:21:082
3425510992119,1cyclictest0-21swapper/221:58:032
3425510992119,1cyclictest0-21swapper/221:58:032
3425510992119,1cyclictest0-21swapper/220:21:372
3425510992119,1cyclictest0-21swapper/200:07:202
3425510992111,8cyclictest3443238-21cron19:59:592
342551099210,20cyclictest0-21swapper/223:22:182
342551099210,20cyclictest0-21swapper/221:09:222
342551099210,20cyclictest0-21swapper/200:33:232
3425505992119,1cyclictest0-21swapper/123:21:351
3425505992119,1cyclictest0-21swapper/120:53:381
3425505992119,1cyclictest0-21swapper/119:38:211
3425505992119,1cyclictest0-21swapper/119:29:431
3425505992119,1cyclictest0-21swapper/100:30:161
3425505992118,2cyclictest3056-21PK-Backend20:02:551
3425505992118,2cyclictest0-21swapper/121:04:191
342550599210,20cyclictest3056-21PK-Backend21:38:261
342550599210,20cyclictest0-21swapper/119:57:141
342550599210,19cyclictest3056-21PK-Backend20:20:191
3425500992117,3cyclictest3471514-21apt21:20:000
3425500992113,6cyclictest3428657-21ntp_states19:15:180
342550099210,20cyclictest0-21swapper/021:44:130
342550099210,20cyclictest0-21swapper/000:30:210
35088752200,10sleep013-21ksoftirqd/023:05:150
3425514992018,1cyclictest0-21swapper/321:34:143
3425514992017,2cyclictest3056-21PK-Backend00:35:163
3425514992017,2cyclictest3056-21PK-Backend00:30:163
3425514992013,5cyclictest3450669-21ls20:20:133
3425514992011,2cyclictest42-21ksoftirqd/322:55:203
3425514992011,2cyclictest42-21ksoftirqd/322:55:203
342551499200,1cyclictest3056-21PK-Backend23:23:443
342551499200,1cyclictest3056-21PK-Backend20:39:253
342551499200,19cyclictest3056-21PK-Backend23:27:023
342551499200,19cyclictest3056-21PK-Backend21:53:073
342551499200,19cyclictest3056-21PK-Backend21:41:443
342551499200,19cyclictest3056-21PK-Backend21:04:133
342551499200,19cyclictest3056-21PK-Backend20:50:543
342551499200,19cyclictest3056-21PK-Backend19:42:073
342551499200,19cyclictest3056-21PK-Backend00:29:423
342551499200,18cyclictest3056-21PK-Backend19:58:023
3425510992018,1cyclictest0-21swapper/223:11:302
3425510992018,1cyclictest0-21swapper/222:15:332
3425510992018,1cyclictest0-21swapper/222:06:482
3425510992018,1cyclictest0-21swapper/220:25:292
3425510992018,1cyclictest0-21swapper/219:26:412
3425510992018,1cyclictest0-21swapper/200:37:282
3425510992014,4cyclictest3495790-21dump-pmu-power22:30:002
3425510992012,6cyclictest883-21dbus-daemon22:50:002
3425510992012,6cyclictest3519497-21sed23:35:202
342551099200,2cyclictest0-21swapper/220:45:292
342551099200,1cyclictest0-21swapper/222:33:592
342551099200,19cyclictest0-21swapper/223:28:002
342551099200,19cyclictest0-21swapper/220:03:452
342551099200,19cyclictest0-21swapper/219:43:462
342551099200,19cyclictest0-21swapper/219:39:532
3425505992018,1cyclictest0-21swapper/123:36:291
3425505992018,1cyclictest0-21swapper/121:51:581
3425505992018,1cyclictest0-21swapper/121:48:201
3425505992018,1cyclictest0-21swapper/120:41:551
3425505992017,2cyclictest3056-21PK-Backend00:17:521
3425505992013,2cyclictest26-21ksoftirqd/100:25:181
3425505992012,5cyclictest1095-21runrttasks20:45:151
342550599200,1cyclictest0-21swapper/122:38:221
342550599200,1cyclictest0-21swapper/119:32:201
342550599200,19cyclictest0-21swapper/123:00:331
342550599200,19cyclictest0-21swapper/122:24:361
342550599200,19cyclictest0-21swapper/122:04:351
342550599200,19cyclictest0-21swapper/121:44:211
342550599200,19cyclictest0-21swapper/120:32:561
342550599200,18cyclictest3056-21PK-Backend23:46:561
3425500992018,1cyclictest0-21swapper/022:04:280
3425500992018,1cyclictest0-21swapper/021:55:290
3425500992018,1cyclictest0-21swapper/021:55:290
3425500992018,1cyclictest0-21swapper/021:48:120
3425500992018,1cyclictest0-21swapper/020:20:590
3425500992018,1cyclictest0-21swapper/000:38:530
3425500992018,1cyclictest0-21swapper/000:07:300
3425500992013,5cyclictest3455640-21cron20:35:000
3425500992013,5cyclictest3442064-21expr19:55:120
3425500992012,6cyclictest436-20systemd-journal19:30:010
3425500992011,7cyclictest3472527-21ntp_states21:20:210
342550099200,1cyclictest0-21swapper/022:10:090
342550099200,19cyclictest3056-21PK-Backend21:10:270
342550099200,19cyclictest0-21swapper/023:35:470
342550099200,19cyclictest0-21swapper/023:32:480
342550099200,19cyclictest0-21swapper/023:29:520
342550099200,19cyclictest0-21swapper/022:36:580
342550099200,19cyclictest0-21swapper/020:37:560
342550099200,19cyclictest0-21swapper/020:27:280
342550099200,19cyclictest0-21swapper/020:13:150
342550099200,19cyclictest0-21swapper/019:51:110
139099200,3rtkit-daemon1389-21rtkit-daemon22:52:521
139099200,14rtkit-daemon0-21swapper/221:44:042
35123682190,4chrt3056-21PK-Backend23:15:173
34467642190,4sleep00-21swapper/020:10:000
34467642190,4sleep00-21swapper/020:10:000
3425514991917,1cyclictest3056-21PK-Backend22:46:203
3425514991917,1cyclictest3056-21PK-Backend20:33:103
3425514991917,1cyclictest3056-21PK-Backend19:45:583
3425514991917,1cyclictest0-21swapper/323:57:453
3425514991917,1cyclictest0-21swapper/323:47:303
3425514991917,1cyclictest0-21swapper/320:15:463
3425514991916,2cyclictest3056-21PK-Backend21:48:133
3425514991914,3cyclictest3535489-21timerwakeupswit00:20:213
3425514991914,3cyclictest2156-21gnome-shell22:40:093
3425514991912,5cyclictest3435577-21ntp_kernel_pll_19:35:193
3425514991911,2cyclictest42-21ksoftirqd/322:40:003
342551499190,9cyclictest3506788-21sort23:00:133
342551499190,1cyclictest3056-21PK-Backend22:22:413
342551499190,1cyclictest3056-21PK-Backend19:27:543
342551499190,1cyclictest0-21swapper/320:55:213
342551499190,1cyclictest0-21swapper/320:55:213
342551499190,1cyclictest0-21swapper/300:15:023
3425510991917,1cyclictest0-21swapper/221:23:562
3425510991917,1cyclictest0-21swapper/221:02:472
3425510991914,4cyclictest978-21snmpd20:06:012
3425510991914,4cyclictest978-21snmpd20:06:012
342551099191,3cyclictest141rcu_preempt22:50:522
3425510991913,4cyclictest34-21ksoftirqd/220:55:142
3425510991913,4cyclictest34-21ksoftirqd/220:55:142
342551099190,1cyclictest3056-21PK-Backend00:25:462
342551099190,1cyclictest0-21swapper/221:38:492
342551099190,18cyclictest0-21swapper/220:31:372
342550599192,12cyclictest3517134-21apt23:30:121
3425505991911,2cyclictest26-21ksoftirqd/119:46:501
3425505991911,2cyclictest26-21ksoftirqd/119:20:121
342550599190,1cyclictest0-21swapper/122:30:241
342550599190,1cyclictest0-21swapper/120:27:491
342550599190,1cyclictest0-21swapper/119:40:451
342550599190,1cyclictest0-21swapper/100:36:451
342550599190,18cyclictest3056-21PK-Backend23:42:101
342550599190,18cyclictest3056-21PK-Backend21:19:541
342550599190,18cyclictest3056-21PK-Backend19:10:161
342550599190,17cyclictest3056-21PK-Backend21:22:451
3425500991917,1cyclictest0-21swapper/023:55:480
3425500991917,1cyclictest0-21swapper/023:51:420
3425500991917,1cyclictest0-21swapper/023:40:130
3425500991917,1cyclictest0-21swapper/023:00:240
3425500991917,1cyclictest0-21swapper/022:21:590
3425500991917,1cyclictest0-21swapper/022:08:540
3425500991917,1cyclictest0-21swapper/021:25:550
3425500991917,1cyclictest0-21swapper/020:41:540
3425500991917,1cyclictest0-21swapper/020:02:340
3425500991917,1cyclictest0-21swapper/019:41:380
3425500991917,1cyclictest0-21swapper/019:20:290
3425500991914,4cyclictest13-21ksoftirqd/000:25:120
3425500991914,3cyclictest3461588-21if_err_enp2s020:50:160
342550099190,1cyclictest0-21swapper/023:16:400
342550099190,1cyclictest0-21swapper/022:16:240
342550099190,1cyclictest0-21swapper/021:35:480
342550099190,1cyclictest0-21swapper/000:24:470
342550099190,18cyclictest0-21swapper/023:12:280
342550099190,18cyclictest0-21swapper/022:32:060
342550099190,18cyclictest0-21swapper/020:19:440
342550099190,18cyclictest0-21swapper/019:35:570
342551499188,8cyclictest42-21ksoftirqd/319:20:203
3425514991816,1cyclictest3056-21PK-Backend22:10:513
3425514991816,1cyclictest0-21swapper/323:53:393
3425514991816,1cyclictest0-21swapper/323:38:373
342551499181,3cyclictest3445831-21memory20:05:173
342551499181,3cyclictest3445831-21memory20:05:173
3425514991813,3cyclictest3521461-21python323:40:213
342551499181,15cyclictest0-21swapper/320:05:013
3425514991810,6cyclictest3491607-21ntp_states22:15:193
342551099186,3cyclictest141rcu_preempt21:55:002
3425510991816,1cyclictest0-21swapper/223:08:122
3425510991816,1cyclictest0-21swapper/220:41:512
3425510991816,1cyclictest0-21swapper/220:37:522
3425510991816,1cyclictest0-21swapper/200:12:002
3425510991812,4cyclictest3490614-21idleruntime-cro22:15:002
342550599182,11cyclictest3468323-21apt21:10:121
3425505991816,1cyclictest0-21swapper/120:55:411
3425505991816,1cyclictest0-21swapper/120:55:411
3425505991811,2cyclictest26-21ksoftirqd/123:54:371
342550599180,1cyclictest3056-21PK-Backend00:23:121
3425500991816,1cyclictest0-21swapper/023:47:360
3425500991816,1cyclictest0-21swapper/022:41:180
3425500991811,6cyclictest3460489-21sshd20:45:260
3425500991811,5cyclictest3464516-21date21:00:000
3425500991811,5cyclictest3464516-21date21:00:000
342551499172,13cyclictest3527529-21/usr/sbin/munin00:00:203
3425514991715,1cyclictest3056-21PK-Backend22:09:363
3425514991714,2cyclictest3056-21PK-Backend23:14:253
3425514991710,5cyclictest3426067-21df19:10:133
342551499170,5cyclictest3453062-21awk20:25:213
3425510991712,4cyclictest3535424-21find00:20:212
3425510991712,3cyclictest3470029-21/usr/sbin/munin21:15:222
3425505991715,1cyclictest3056-21PK-Backend20:05:181
3425505991715,1cyclictest3056-21PK-Backend20:05:181
342550599171,11cyclictest3500998-21apt-get22:45:041
342550599170,1cyclictest3056-21PK-Backend21:34:001
3425500991715,1cyclictest0-21swapper/022:28:070
3425500991713,3cyclictest3466672-21cpuspeed21:05:130
3425500991711,5cyclictest3433714-21ls19:30:170
3425500991710,5cyclictest3465242-21grep21:00:180
34664132160,2sleep30-21swapper/321:05:013
3425514991614,1cyclictest3056-21PK-Backend23:09:183
3425514991613,2cyclictest3056-21PK-Backend00:05:553
3425514991612,2cyclictest42-21ksoftirqd/323:30:183
342551499160,15cyclictest3477002-21apt21:35:113
3425510991614,1cyclictest3056-21PK-Backend22:36:422
3425510991612,3cyclictest3440723-21needreboot19:50:182
342550599161,11cyclictest3489995-21python322:10:211
342550099168,6cyclictest436-20systemd-journal23:25:000
342550099167,3cyclictest141rcu_preempt19:45:130
3425500991614,1cyclictest0-21swapper/021:31:220
3425500991612,3cyclictest3527732-21cpuspeed_turbos00:00:120
342551499159,4cyclictest3484438-21cut21:55:163
342551499159,4cyclictest3484438-21cut21:55:163
342551499154,2cyclictest141rcu_preempt22:32:043
3425514991513,1cyclictest3056-21PK-Backend22:01:383
3425510991513,1cyclictest0-21swapper/223:30:022
3425510991513,1cyclictest0-21swapper/219:30:332
342550599157,2cyclictest141rcu_preempt20:12:581
3425505991511,2cyclictest26-21ksoftirqd/121:25:161
342550099159,5cyclictest3505517-21ls22:55:160
342550099159,5cyclictest3505517-21ls22:55:160
3425500991511,3cyclictest1095-21runrttasks19:10:190
3425514991412,1cyclictest3056-21PK-Backend00:11:303
342551499140,12cyclictest3056-21PK-Backend19:15:013
342551099148,5cyclictest3505347-21expr22:55:122
342551099148,5cyclictest3505347-21expr22:55:122
342551099148,4cyclictest978-21snmpd19:17:032
342550099144,1cyclictest141rcu_preempt00:15:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional