You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-21 - 08:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sun Jun 21, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
308688021290,4sleep23056-21PK-Backend19:30:002
308505521100,3sleep33056-21PK-Backend19:25:003
307876927740,31sleep00-21swapper/019:08:460
307884427144,18sleep10-21swapper/119:09:441
307814127149,13sleep30-21swapper/319:05:243
307884026746,14sleep20-21swapper/219:09:412
30997272400,2sleep30-21swapper/320:05:153
31384812270,4chrt3056-21PK-Backend21:55:201
139099260,4rtkit-daemon1389-21rtkit-daemon00:28:430
307918999257,15cyclictest3165772-21unin-run23:15:001
3079182992517,6cyclictest3121858-21cron21:10:000
31590172230,7sleep234-21ksoftirqd/222:55:032
3079189992316,5cyclictest3107727-21idleruntime-cro20:30:001
3079189992314,7cyclictest3128398-21sshd21:25:231
31800582220,8sleep126-21ksoftirqd/123:55:001
3079194992220,1cyclictest0-21swapper/319:43:383
3079194992219,2cyclictest3056-21PK-Backend19:26:213
307919499220,20cyclictest3056-21PK-Backend21:22:533
307919499220,20cyclictest3056-21PK-Backend21:22:533
307919499220,20cyclictest3056-21PK-Backend21:05:173
3079191992220,1cyclictest0-21swapper/221:45:542
3079191992220,1cyclictest0-21swapper/221:05:102
3079191992220,1cyclictest0-21swapper/221:03:212
3079191992220,1cyclictest0-21swapper/200:18:222
3079191992220,1cyclictest0-21swapper/200:13:432
3079191992217,3cyclictest436-20systemd-journal23:34:322
3079191992216,5cyclictest3118331-21pool-whoopsie20:59:292
3079189992211,9cyclictest3151364-21unin-run22:34:591
307918999220,20cyclictest3080632-21taskset19:10:581
3079182992213,7cyclictest3176403-21cron23:44:590
307918299220,21cyclictest0-21swapper/021:37:010
139099220,13rtkit-daemon0-21swapper/019:52:130
31677222210,5sleep342-21ksoftirqd/323:20:113
31677222210,5sleep342-21ksoftirqd/323:20:113
3079194992119,1cyclictest3056-21PK-Backend23:12:103
3079194992119,1cyclictest3056-21PK-Backend21:02:333
3079194992119,1cyclictest3056-21PK-Backend20:53:063
3079194992119,1cyclictest0-21swapper/321:56:413
3079194992119,1cyclictest0-21swapper/321:47:413
3079194992119,1cyclictest0-21swapper/321:30:253
3079194992119,1cyclictest0-21swapper/319:31:143
3079194992119,1cyclictest0-21swapper/300:06:183
3079194992118,2cyclictest3056-21PK-Backend20:32:113
307919499211,17cyclictest3079474-21apt19:10:113
307919499210,20cyclictest3056-21PK-Backend20:38:333
307919499210,20cyclictest3056-21PK-Backend19:55:203
307919499210,20cyclictest0-21swapper/300:16:463
307919499210,19cyclictest3056-21PK-Backend23:38:403
307919499210,19cyclictest3056-21PK-Backend23:09:553
307919499210,19cyclictest3056-21PK-Backend22:51:363
307919499210,19cyclictest3056-21PK-Backend22:37:513
307919499210,19cyclictest3056-21PK-Backend22:22:353
307919499210,19cyclictest3056-21PK-Backend21:40:593
3079191992119,1cyclictest0-21swapper/223:54:232
3079191992119,1cyclictest0-21swapper/223:49:222
3079191992119,1cyclictest0-21swapper/223:12:122
3079191992119,1cyclictest0-21swapper/223:08:482
3079191992119,1cyclictest0-21swapper/222:53:392
3079191992119,1cyclictest0-21swapper/222:01:482
3079191992119,1cyclictest0-21swapper/221:39:112
3079191992119,1cyclictest0-21swapper/220:20:092
3079191992119,1cyclictest0-21swapper/220:02:372
3079191992119,1cyclictest0-21swapper/219:13:042
3079191992119,1cyclictest0-21swapper/200:31:012
3079191992113,6cyclictest3154314-21ntp_kernel_pll_22:40:182
307919199210,20cyclictest0-21swapper/223:25:102
307919199210,20cyclictest0-21swapper/222:32:312
307919199210,20cyclictest0-21swapper/222:06:082
307919199210,20cyclictest0-21swapper/200:28:172
307919199210,1cyclictest0-21swapper/221:25:532
3079189992114,6cyclictest3131431-21needreboot21:35:171
3079189992110,8cyclictest883-21dbus-daemon22:45:011
307918999210,20cyclictest0-21swapper/120:47:171
307918999210,19cyclictest3056-21PK-Backend22:03:421
307918999210,19cyclictest3056-21PK-Backend19:40:051
3079182992119,1cyclictest0-21swapper/021:31:270
3079182992113,6cyclictest3106647-21awk20:25:140
3079182992113,6cyclictest13-21ksoftirqd/019:10:050
307918299210,20cyclictest0-21swapper/023:14:210
307918299210,20cyclictest0-21swapper/022:33:120
307918299210,20cyclictest0-21swapper/021:43:440
307918299210,20cyclictest0-21swapper/020:45:230
307918299210,20cyclictest0-21swapper/019:32:370
139099210,3rtkit-daemon1389-21rtkit-daemon00:12:160
139099210,3rtkit-daemon1389-21rtkit-daemon00:08:300
139099210,14rtkit-daemon0-21swapper/019:57:500
139099210,14rtkit-daemon0-21swapper/019:45:220
3079194992018,1cyclictest3056-21PK-Backend23:56:533
3079194992018,1cyclictest3056-21PK-Backend23:44:353
3079194992018,1cyclictest3056-21PK-Backend23:31:043
3079194992018,1cyclictest3056-21PK-Backend00:37:413
3079194992018,1cyclictest0-21swapper/322:07:563
3079194992014,4cyclictest3089185-21cat19:35:173
307919499200,1cyclictest3056-21PK-Backend21:18:213
307919499200,19cyclictest3056-21PK-Backend23:00:283
307919499200,19cyclictest3056-21PK-Backend22:43:513
307919499200,19cyclictest3056-21PK-Backend20:44:353
307919499200,19cyclictest3056-21PK-Backend20:03:593
307919499200,19cyclictest3056-21PK-Backend19:47:433
307919499200,19cyclictest3056-21PK-Backend00:04:233
307919499200,19cyclictest0-21swapper/300:25:523
307919499200,19cyclictest0-21swapper/300:13:283
3079191992018,1cyclictest0-21swapper/223:35:172
3079191992018,1cyclictest0-21swapper/221:32:072
3079191992018,1cyclictest0-21swapper/220:45:372
3079191992018,1cyclictest0-21swapper/220:37:522
3079191992018,1cyclictest0-21swapper/220:11:442
3079191992018,1cyclictest0-21swapper/200:09:042
3079191992018,1cyclictest0-21swapper/200:02:422
307919199200,1cyclictest0-21swapper/220:25:432
307919199200,1cyclictest0-21swapper/220:00:012
307919199200,1cyclictest0-21swapper/219:34:322
307919199200,19cyclictest0-21swapper/223:43:292
307919199200,19cyclictest0-21swapper/221:51:072
307919199200,19cyclictest0-21swapper/220:54:502
3079189992018,1cyclictest0-21swapper/123:58:131
3079189992018,1cyclictest0-21swapper/122:53:251
3079189992018,1cyclictest0-21swapper/120:44:271
3079189992018,1cyclictest0-21swapper/120:30:221
3079189992018,1cyclictest0-21swapper/120:07:101
3079189992017,2cyclictest3090125-21sh19:39:591
3079189992017,2cyclictest3056-21PK-Backend00:12:201
307918999200,1cyclictest0-21swapper/121:20:171
307918999200,1cyclictest0-21swapper/121:20:171
307918999200,19cyclictest0-21swapper/123:49:561
307918999200,19cyclictest0-21swapper/122:56:141
307918999200,19cyclictest0-21swapper/121:10:501
307918999200,19cyclictest0-21swapper/120:19:321
307918999200,19cyclictest0-21swapper/119:53:581
307918999200,19cyclictest0-21swapper/119:28:301
307918999200,19cyclictest0-21swapper/119:21:341
307918999200,18cyclictest0-21swapper/123:20:081
307918999200,18cyclictest0-21swapper/123:20:071
3079182992018,1cyclictest0-21swapper/023:55:520
3079182992018,1cyclictest0-21swapper/023:38:480
3079182992018,1cyclictest0-21swapper/023:34:010
3079182992018,1cyclictest0-21swapper/023:29:500
3079182992018,1cyclictest0-21swapper/023:00:230
3079182992018,1cyclictest0-21swapper/022:58:400
3079182992018,1cyclictest0-21swapper/020:51:040
3079182992018,1cyclictest0-21swapper/020:22:180
3079182992018,1cyclictest0-21swapper/020:11:170
3079182992018,1cyclictest0-21swapper/020:09:410
3079182992018,1cyclictest0-21swapper/019:41:160
3079182992018,1cyclictest0-21swapper/019:29:050
3079182992015,3cyclictest978-21snmpd21:19:480
3079182992013,5cyclictest3110311-21cat20:35:120
3079182992012,6cyclictest883-21dbus-daemon23:09:300
3079182992012,6cyclictest436-20systemd-journal21:30:000
307918299200,1cyclictest0-21swapper/000:24:250
307918299200,1cyclictest0-21swapper/000:24:250
307918299200,1cyclictest0-21swapper/000:19:250
307918299200,19cyclictest0-21swapper/022:22:170
307918299200,19cyclictest0-21swapper/022:11:550
307918299200,19cyclictest0-21swapper/020:56:040
307918299200,19cyclictest0-21swapper/020:02:110
307918299200,19cyclictest0-21swapper/019:38:180
139099200,14rtkit-daemon0-21swapper/022:44:120
139099200,13rtkit-daemon0-21swapper/021:14:200
307919499198,8cyclictest42-21ksoftirqd/319:50:193
3079194991917,1cyclictest3056-21PK-Backend23:54:573
3079194991917,1cyclictest3056-21PK-Backend23:25:093
3079194991917,1cyclictest3056-21PK-Backend22:59:543
3079194991917,1cyclictest3056-21PK-Backend22:12:493
3079194991917,1cyclictest0-21swapper/322:00:263
3079194991916,2cyclictest3056-21PK-Backend22:45:343
307919499190,1cyclictest3056-21PK-Backend22:32:233
307919499190,1cyclictest3056-21PK-Backend22:26:563
307919499190,1cyclictest3056-21PK-Backend20:59:483
307919499190,1cyclictest3056-21PK-Backend20:26:563
307919499190,1cyclictest0-21swapper/321:52:543
307919499190,1cyclictest0-21swapper/300:30:313
307919499190,1cyclictest0-21swapper/300:24:313
307919499190,1cyclictest0-21swapper/300:24:313
307919499190,18cyclictest3056-21PK-Backend20:46:513
307919499190,18cyclictest3056-21PK-Backend19:17:013
307919499190,18cyclictest3056-21PK-Backend19:17:003
307919199195,2cyclictest141rcu_preempt20:40:142
3079191991917,1cyclictest0-21swapper/223:58:162
3079191991917,1cyclictest0-21swapper/222:26:282
3079191991917,1cyclictest0-21swapper/222:12:102
3079191991917,1cyclictest0-21swapper/220:33:332
3079191991917,1cyclictest0-21swapper/219:38:182
3079191991912,6cyclictest3094835-21ntpq19:50:192
307919199190,1cyclictest0-21swapper/221:43:172
307919199190,1cyclictest0-21swapper/221:13:432
307919199190,1cyclictest0-21swapper/220:19:352
307919199190,1cyclictest0-21swapper/200:21:412
307919199190,1cyclictest0-21swapper/200:21:402
307919199190,18cyclictest0-21swapper/222:24:462
307919199190,18cyclictest0-21swapper/221:57:292
3079189991917,1cyclictest3056-21PK-Backend22:07:271
3079189991917,1cyclictest0-21swapper/120:56:171
3079189991917,1cyclictest0-21swapper/120:20:081
3079189991916,2cyclictest3056-21PK-Backend21:49:301
3079189991913,4cyclictest3162267-21idleruntime-cro23:05:001
3079189991912,6cyclictest886-21NetworkManager23:34:311
307918999190,1cyclictest3056-21PK-Backend19:30:401
307918999190,1cyclictest3056-21PK-Backend00:31:061
307918999190,1cyclictest3056-21PK-Backend00:16:121
307918999190,1cyclictest3056-21PK-Backend00:07:061
307918999190,1cyclictest0-21swapper/123:09:051
307918999190,1cyclictest0-21swapper/122:15:331
307918999190,1cyclictest0-21swapper/121:40:441
307918999190,1cyclictest0-21swapper/121:02:051
307918999190,1cyclictest0-21swapper/120:54:281
307918999190,1cyclictest0-21swapper/120:37:181
307918999190,1cyclictest0-21swapper/100:00:091
307918999190,18cyclictest0-21swapper/123:39:151
307918999190,18cyclictest0-21swapper/122:26:281
307918999190,18cyclictest0-21swapper/121:34:371
307918999190,18cyclictest0-21swapper/119:47:431
3079182991917,1cyclictest0-21swapper/022:06:080
3079182991917,1cyclictest0-21swapper/022:02:570
3079182991917,1cyclictest0-21swapper/021:53:020
3079182991917,1cyclictest0-21swapper/000:32:080
3079182991916,2cyclictest0-21swapper/022:50:220
3079182991914,3cyclictest3168654-21switchtime23:20:210
3079182991914,3cyclictest3168654-21switchtime23:20:210
307918299190,1cyclictest0-21swapper/020:42:330
307918299190,18cyclictest0-21swapper/022:39:120
307918299190,18cyclictest0-21swapper/022:27:440
307918299190,18cyclictest0-21swapper/021:59:460
307918299190,18cyclictest0-21swapper/020:32:120
139099190,13rtkit-daemon0-21swapper/020:19:020
139099190,13rtkit-daemon0-21swapper/000:01:110
3079194991816,1cyclictest0-21swapper/323:15:353
307919199183,4cyclictest34-21ksoftirqd/200:35:152
3079191991816,1cyclictest0-21swapper/222:35:222
3079191991816,1cyclictest0-21swapper/222:18:112
3079191991816,1cyclictest0-21swapper/221:24:582
3079191991816,1cyclictest0-21swapper/221:24:582
3079191991816,1cyclictest0-21swapper/221:17:422
3079191991813,3cyclictest3167931-21tr23:20:112
3079191991813,3cyclictest3167931-21tr23:20:112
3079191991813,3cyclictest2156-21gnome-shell23:18:092
3079189991816,1cyclictest3056-21PK-Backend21:50:051
3079189991816,1cyclictest0-21swapper/123:44:011
3079189991816,1cyclictest0-21swapper/122:45:411
3079189991816,1cyclictest0-21swapper/120:01:151
3079189991816,1cyclictest0-21swapper/119:17:011
3079189991816,1cyclictest0-21swapper/119:17:011
3079189991816,1cyclictest0-21swapper/100:38:161
307918999181,12cyclictest3151603-21apt22:35:101
3079189991810,6cyclictest918-21rs:main1
3079182991816,1cyclictest0-21swapper/022:48:390
3079182991816,1cyclictest0-21swapper/021:20:380
3079182991816,1cyclictest0-21swapper/021:20:380
3079182991812,5cyclictest3082633-21cat19:15:200
3079182991812,5cyclictest3082633-21cat19:15:200
3079182991812,4cyclictest1095-21runrttasks23:45:210
3079182991811,6cyclictest3085111-21sed19:25:010
3079182991811,5cyclictest3166344-21expr23:15:130
3079194991715,1cyclictest3056-21PK-Backend21:11:183
3079194991712,3cyclictest3128229-21switchtime21:25:213
3079191991710,5cyclictest436-20systemd-journal19:47:232
3079191991710,5cyclictest3090590-21touch19:40:012
307918999179,6cyclictest3146721-21expr22:20:141
307918999172,11cyclictest3143511-21runrttasks22:10:191
3079189991715,1cyclictest3056-21PK-Backend23:15:221
3079189991715,1cyclictest3056-21PK-Backend00:29:091
3079189991715,1cyclictest0-21swapper/123:29:401
3079189991712,4cyclictest26-21ksoftirqd/121:05:211
3079182991715,1cyclictest0-21swapper/023:53:560
3079182991715,1cyclictest0-21swapper/000:36:000
3079182991714,2cyclictest3145365-21/usr/sbin/munin22:15:170
307919499160,14cyclictest3131646-21chrt21:35:213
3079191991614,1cyclictest0-21swapper/222:45:222
3079191991614,1cyclictest0-21swapper/220:07:102
3079191991612,2cyclictest3081862-21tr19:15:122
3079191991612,2cyclictest3081862-21tr19:15:122
3079189991614,1cyclictest3056-21PK-Backend00:23:551
3079189991614,1cyclictest3056-21PK-Backend00:23:551
3079189991612,3cyclictest3101503-21if_enp2s020:10:151
307918299169,5cyclictest3120207-21cat21:05:010
307918299163,2cyclictest141rcu_preempt21:45:140
307919499158,5cyclictest3145574-21grep22:15:203
3079194991513,1cyclictest3056-21PK-Backend20:12:173
3079194991511,3cyclictest3177397-21grep23:45:173
3079191991511,3cyclictest3084944-21idleruntime-cro19:25:002
307919199151,12cyclictest3160922-21tr23:00:122
307918999159,4cyclictest3097405-21latency_hist20:00:011
307919499149,4cyclictest42-21ksoftirqd/320:20:173
307919499139,3cyclictest3056-21PK-Backend20:15:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional