You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-16 - 12:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Jul 16, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
58950421130,3sleep342-21ksoftirqd/323:46:193
49054326849,14sleep10-21swapper/119:06:321
49058426646,14sleep20-21swapper/219:07:062
5713002640,3sleep1141rcu_preempt22:55:131
49065326344,13sleep30-21swapper/319:07:593
48932526145,9sleep00-21swapper/019:05:060
5465072450,4chrt3056-21PK-Backend21:45:111
5962142420,3chrt3056-21PK-Backend00:05:221
490929992910,17cyclictest533761-21turbostat21:10:000
5448442280,4sleep0544849-21cpuspeed_turbos21:40:130
49092999270,25cyclictest0-21swapper/022:20:150
490929992610,15cyclictest3056-21PK-Backend21:55:490
490929992610,15cyclictest3056-21PK-Backend21:55:490
49092999252,9cyclictest562857-21cut22:30:210
49092999252,7cyclictest494031-21head19:15:150
49092999250,19cyclictest0-21swapper/019:46:240
49092999246,5cyclictest510459-21cat20:00:210
49092999246,5cyclictest510459-21cat20:00:210
49092999244,5cyclictest504198-21cut19:45:010
49092999241,8cyclictest534692-21strings21:10:180
490939992313,8cyclictest883-21dbus-daemon23:52:163
490936992321,1cyclictest0-21swapper/223:12:282
49093699230,21cyclictest1095-21runrttasks19:31:172
49092999238,4cyclictest500293-21cron19:39:550
49092999238,3cyclictest506462-21if_enp2s019:50:160
49092999238,3cyclictest498414-21dpkg19:30:010
49092999237,3cyclictest574195-21munin-plugin-st23:05:000
490929992321,1cyclictest0-21swapper/021:26:010
49092999231,19cyclictest496581-21munin-run19:25:010
490929992311,10cyclictest549610-21apt21:54:590
490929992310,3cyclictest1095-21runrttasks20:40:400
49092999230,8cyclictest536696-21find21:15:210
49092999230,21cyclictest3056-21PK-Backend00:25:140
490939992220,1cyclictest0-21swapper/320:15:093
490939992219,2cyclictest3056-21PK-Backend20:20:453
490939992219,2cyclictest0-21swapper/320:11:313
490939992216,4cyclictest42-21ksoftirqd/323:20:003
490939992212,4cyclictest42-21ksoftirqd/300:19:203
49093999221,19cyclictest569855-21python322:50:193
49093999221,19cyclictest569855-21python322:50:193
49093999220,21cyclictest0-21swapper/319:29:073
49093999220,20cyclictest564853-21timerwakeupswit22:35:233
490936992220,1cyclictest0-21swapper/222:22:332
490936992220,1cyclictest0-21swapper/222:16:322
490936992220,1cyclictest0-21swapper/222:07:182
490936992220,1cyclictest0-21swapper/220:34:322
490936992220,1cyclictest0-21swapper/220:26:482
490936992220,1cyclictest0-21swapper/219:10:162
490936992220,1cyclictest0-21swapper/200:37:382
490936992213,8cyclictest589505-21sshd23:46:192
490932992220,1cyclictest3056-21PK-Backend22:43:461
490932992220,1cyclictest3056-21PK-Backend22:36:361
490932992220,1cyclictest3056-21PK-Backend19:34:401
490932992219,2cyclictest3056-21PK-Backend20:12:531
49093299220,21cyclictest3056-21PK-Backend23:52:541
49093299220,21cyclictest3056-21PK-Backend20:43:491
49093299220,20cyclictest0-21swapper/120:56:321
49092999229,2cyclictest579643-21/usr/sbin/munin23:20:130
49092999229,11cyclictest576252-21cpuspeed23:10:110
49092999228,4cyclictest547183-21open_inodes21:45:200
49092999228,3cyclictest0-21swapper/022:10:200
49092999227,2cyclictest492621-21turbostat19:15:000
49092999225,5cyclictest564348-21cat22:35:160
490929992220,1cyclictest0-21swapper/023:09:160
490929992220,1cyclictest0-21swapper/020:32:070
490929992215,6cyclictest522763-21python320:35:210
49092999221,3cyclictest0-21swapper/020:55:120
49092999220,7cyclictest537605-21/usr/sbin/munin21:20:130
490939992119,1cyclictest3056-21PK-Backend21:10:253
490939992119,1cyclictest0-21swapper/323:30:523
490939992119,1cyclictest0-21swapper/322:34:143
490939992119,1cyclictest0-21swapper/322:07:313
490939992119,1cyclictest0-21swapper/321:58:313
490939992119,1cyclictest0-21swapper/321:58:313
490939992119,1cyclictest0-21swapper/321:06:403
490939992119,1cyclictest0-21swapper/320:01:453
490939992119,1cyclictest0-21swapper/320:01:443
490939992119,1cyclictest0-21swapper/319:30:423
490939992118,2cyclictest4176944-21snapd21:36:133
490939992113,6cyclictest547158-21ntp_states21:45:183
49093999211,17cyclictest557670-21python322:15:203
49093999211,17cyclictest2156-21gnome-shell23:21:093
49093999210,20cyclictest0-21swapper/321:43:033
49093999210,18cyclictest978-21snmpd23:05:113
490936992119,1cyclictest0-21swapper/223:59:462
490936992119,1cyclictest0-21swapper/223:54:462
490936992119,1cyclictest0-21swapper/223:29:592
490936992119,1cyclictest0-21swapper/221:52:312
490936992119,1cyclictest0-21swapper/221:47:522
490936992119,1cyclictest0-21swapper/221:38:122
490936992119,1cyclictest0-21swapper/220:45:462
490936992119,1cyclictest0-21swapper/220:00:042
490936992119,1cyclictest0-21swapper/220:00:032
490936992119,1cyclictest0-21swapper/200:20:482
49093699210,9cyclictest5243742sh20:40:182
49093699210,20cyclictest0-21swapper/221:41:432
49093699210,20cyclictest0-21swapper/221:02:512
49093699210,20cyclictest0-21swapper/220:20:252
49093699210,20cyclictest0-21swapper/220:18:292
49093699210,20cyclictest0-21swapper/200:19:272
490932992119,1cyclictest3056-21PK-Backend23:39:161
490932992119,1cyclictest3056-21PK-Backend22:30:011
490932992119,1cyclictest3056-21PK-Backend22:20:151
490932992119,1cyclictest3056-21PK-Backend22:15:541
490932992119,1cyclictest3056-21PK-Backend20:30:231
490932992119,1cyclictest0-21swapper/120:01:231
490932992119,1cyclictest0-21swapper/120:01:231
490932992118,2cyclictest0-21swapper/121:25:181
490932992113,6cyclictest536497-21ntp_states21:15:171
49093299210,20cyclictest3056-21PK-Backend21:44:321
49093299210,20cyclictest0-21swapper/121:14:111
49093299210,20cyclictest0-21swapper/121:04:571
49093299210,19cyclictest3056-21PK-Backend00:38:101
49092999219,10cyclictest570458-21latency_hist22:55:000
49092999219,10cyclictest570458-21latency_hist22:55:000
49092999217,4cyclictest1095-21runrttasks23:27:570
49092999217,2cyclictest601955-21cron00:25:000
49092999216,4cyclictest604756-21grep00:30:180
49092999213,4cyclictest527287-21cut20:50:140
49092999211,9cyclictest588083-21munin-run23:45:000
490929992119,1cyclictest0-21swapper/023:59:520
490929992119,1cyclictest0-21swapper/023:54:330
490929992119,1cyclictest0-21swapper/023:32:550
49092999211,8cyclictest1788620-21WebKitWebProces19:55:240
490929992116,4cyclictest0-21swapper/021:03:370
49092999210,8cyclictest597784-21sshd00:10:200
49092999210,20cyclictest0-21swapper/022:42:460
49092999210,20cyclictest0-21swapper/020:18:150
49092999210,20cyclictest0-21swapper/020:14:360
5811322200,4sleep13056-21PK-Backend23:25:001
49093999208,4cyclictest42-21ksoftirqd/322:11:113
490939992018,1cyclictest3056-21PK-Backend19:52:313
490939992018,1cyclictest0-21swapper/322:49:423
490939992018,1cyclictest0-21swapper/322:44:483
490939992018,1cyclictest0-21swapper/321:33:033
490939992018,1cyclictest0-21swapper/320:07:533
490939992018,1cyclictest0-21swapper/300:35:473
490939992018,1cyclictest0-21swapper/300:26:083
490939992018,1cyclictest0-21swapper/300:06:133
490939992018,1cyclictest0-21swapper/300:01:133
490939992015,4cyclictest42-21ksoftirqd/323:40:013
490939992015,3cyclictest491885-21ntp_states19:10:193
490939992012,3cyclictest42-21ksoftirqd/300:13:493
49093999201,17cyclictest553107-21munin-run22:05:003
490939992010,4cyclictest42-21ksoftirqd/323:26:213
490939992010,2cyclictest141rcu_preempt00:30:433
49093999200,19cyclictest3056-21PK-Backend19:39:473
49093999200,19cyclictest0-21swapper/322:56:183
49093999200,19cyclictest0-21swapper/322:26:303
49093999200,19cyclictest0-21swapper/322:21:023
49093999200,19cyclictest0-21swapper/321:51:153
49093999200,19cyclictest0-21swapper/320:52:493
49093999200,18cyclictest3056-21PK-Backend21:18:303
490936992018,1cyclictest0-21swapper/223:42:502
490936992018,1cyclictest0-21swapper/223:38:242
490936992018,1cyclictest0-21swapper/223:04:512
490936992018,1cyclictest0-21swapper/222:33:072
490936992018,1cyclictest0-21swapper/222:11:382
490936992018,1cyclictest0-21swapper/221:11:282
490936992018,1cyclictest0-21swapper/220:57:362
490936992018,1cyclictest0-21swapper/219:40:382
490936992018,1cyclictest0-21swapper/200:07:182
490936992017,2cyclictest0-21swapper/219:25:222
490936992013,5cyclictest541950-21cat21:30:202
490936992012,6cyclictest604821-21sensors_temp00:30:202
490936992011,7cyclictest580035-21df_inode23:20:132
49093699200,1cyclictest0-21swapper/219:21:372
49093699200,1cyclictest0-21swapper/200:12:172
49093699200,19cyclictest3056-21PK-Backend21:06:352
49093699200,19cyclictest0-21swapper/222:45:592
49093699200,19cyclictest0-21swapper/222:03:332
49093699200,19cyclictest0-21swapper/221:55:092
49093699200,19cyclictest0-21swapper/221:55:082
49093699200,19cyclictest0-21swapper/220:05:042
49093699200,19cyclictest0-21swapper/219:58:072
49093699200,19cyclictest0-21swapper/200:02:162
490932992018,1cyclictest3056-21PK-Backend20:28:271
490932992018,1cyclictest0-21swapper/119:10:061
490932992013,5cyclictest538193-21ntp_kernel_err21:20:181
49093299200,2cyclictest3056-21PK-Backend23:01:371
49093299200,1cyclictest3056-21PK-Backend23:47:201
49093299200,1cyclictest3056-21PK-Backend22:47:311
49093299200,1cyclictest3056-21PK-Backend00:27:221
49093299200,19cyclictest3056-21PK-Backend23:44:511
49093299200,19cyclictest3056-21PK-Backend21:57:221
49093299200,19cyclictest3056-21PK-Backend21:57:221
49093299200,19cyclictest3056-21PK-Backend20:16:301
49093299200,19cyclictest3056-21PK-Backend19:55:141
49093299200,19cyclictest0-21swapper/121:05:311
49093299200,19cyclictest0-21swapper/119:36:151
49092999205,4cyclictest13-21ksoftirqd/022:10:010
49092999201,8cyclictest0-21swapper/020:05:500
490929992018,1cyclictest0-21swapper/023:36:200
490929992018,1cyclictest0-21swapper/022:02:510
490929992018,1cyclictest0-21swapper/021:33:110
490929992018,1cyclictest0-21swapper/000:15:130
490929992018,1cyclictest0-21swapper/000:06:350
49092999201,7cyclictest544426-21munin-run21:40:000
49092999201,5cyclictest578789-21python323:15:200
49092999200,8cyclictest567237-21/usr/sbin/munin22:45:230
49092999200,7cyclictest605838-21tr00:35:130
49092999200,7cyclictest517547-21/usr/sbin/munin20:20:210
49092999200,2cyclictest0-21swapper/022:57:340
49092999200,19cyclictest0-21swapper/023:46:410
49092999200,19cyclictest0-21swapper/020:46:130
49093999192,13cyclictest576199-21apt23:10:103
490939991917,1cyclictest3056-21PK-Backend19:15:203
490939991917,1cyclictest0-21swapper/321:00:523
490939991917,1cyclictest0-21swapper/320:40:033
490939991917,1cyclictest0-21swapper/320:38:423
490939991916,2cyclictest3056-21PK-Backend19:59:153
490939991914,3cyclictest520523-21df_inode20:30:123
490939991911,6cyclictest526157-21wc20:45:213
49093999190,1cyclictest3056-21PK-Backend19:23:113
49093999190,1cyclictest0-21swapper/323:44:173
49093999190,1cyclictest0-21swapper/320:28:143
49093999190,1cyclictest0-21swapper/300:22:513
49093999190,18cyclictest0-21swapper/323:56:143
49093999190,18cyclictest0-21swapper/320:57:423
49093699196,11cyclictest603700-21cron00:30:002
490936991917,1cyclictest0-21swapper/222:35:242
490936991917,1cyclictest0-21swapper/219:50:512
490936991917,1cyclictest0-21swapper/219:38:142
490936991911,6cyclictest574846-21fw_packets23:05:162
49093699190,1cyclictest0-21swapper/222:44:512
49093699190,1cyclictest0-21swapper/221:29:472
49093699190,18cyclictest0-21swapper/221:16:222
49093699190,18cyclictest0-21swapper/220:52:082
49093699190,18cyclictest0-21swapper/220:37:502
490932991917,1cyclictest3056-21PK-Backend20:38:071
490932991917,1cyclictest3056-21PK-Backend00:13:171
490932991917,1cyclictest3056-21PK-Backend00:00:591
490932991911,6cyclictest528552-21cron20:54:591
49093299190,1cyclictest3056-21PK-Backend22:11:351
49093299190,1cyclictest3056-21PK-Backend22:02:361
49093299190,1cyclictest3056-21PK-Backend20:49:021
49093299190,1cyclictest0-21swapper/123:55:251
49093299190,1cyclictest0-21swapper/120:24:351
49093299190,1cyclictest0-21swapper/119:51:371
49093299190,1cyclictest0-21swapper/119:22:021
490929991917,1cyclictest0-21swapper/022:17:040
49092999190,9cyclictest499584-21ntp_states19:30:190
49092999190,18cyclictest0-21swapper/000:03:310
49093999189,3cyclictest42-21ksoftirqd/323:01:033
490939991816,1cyclictest3056-21PK-Backend21:20:473
490939991816,1cyclictest3056-21PK-Backend19:42:113
49093999180,5cyclictest3056-21PK-Backend21:26:103
490936991816,1cyclictest0-21swapper/222:58:012
490936991816,1cyclictest0-21swapper/222:52:002
490936991816,1cyclictest0-21swapper/222:52:002
490936991814,3cyclictest978-21snmpd23:34:212
490932991812,5cyclictest26-21ksoftirqd/119:15:021
49092999180,7cyclictest560486-21cat22:25:120
490939991714,2cyclictest3056-21PK-Backend19:45:223
49093699172,14cyclictest1095-21runrttasks23:18:212
490936991715,1cyclictest0-21swapper/220:12:202
490936991711,5cyclictest493809-21awk19:15:132
490932991715,1cyclictest3056-21PK-Backend22:05:471
490932991715,1cyclictest3056-21PK-Backend20:08:061
490932991715,1cyclictest3056-21PK-Backend00:33:441
490932991715,1cyclictest0-21swapper/121:50:401
490932991712,3cyclictest570452-21sh22:55:011
490932991712,3cyclictest570452-21sh22:55:001
490932991711,5cyclictest505282-21ls19:45:221
490932991710,5cyclictest26-21ksoftirqd/119:25:221
49092999170,1cyclictest0-21swapper/020:28:150
490936991610,5cyclictest560946-21strings22:25:172
49093299169,5cyclictest502879-21cat19:40:121
490932991614,1cyclictest3056-21PK-Backend23:15:301
490932991614,1cyclictest3056-21PK-Backend00:22:561
490932991612,3cyclictest82332-21update-manager23:07:151
49093699159,4cyclictest504783-21cut19:45:162
49093299153,10cyclictest543679-21awk21:35:201
490932991412,1cyclictest3056-21PK-Backend22:30:351
490932991411,2cyclictest3056-21PK-Backend21:33:361
49093299139,3cyclictest3056-21PK-Backend23:14:211
49093299139,2cyclictest584621-21perf23:35:001
490932991311,1cyclictest3056-21PK-Backend00:16:341
49093699120,4cyclictest141rcu_preempt21:20:202
49093299116,3cyclictest5813662sleep123:25:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional