You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-24 - 06:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Mon Aug 24, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
151925921230,2sleep33056-21PK-Backend21:10:203
15158302990,3sleep20-21swapper/221:00:202
147554727639,14sleep30-21swapper/319:07:263
139099760,62rtkit-daemon0-21swapper/119:07:561
147548727539,14sleep00-21swapper/019:06:380
147566427437,32sleep20-21swapper/219:08:582
15736832480,3sleep30-21swapper/323:45:173
139099360,4rtkit-daemon1788643-21eadedCompositor21:50:400
139099360,2rtkit-daemon0-21swapper/000:15:170
139099350,4rtkit-daemon1788643-21eadedCompositor20:09:560
139099340,4rtkit-daemon2363-21evolution-alarm22:47:540
15571002290,4sleep00-21swapper/023:00:120
1475907992810,16cyclictest1566598-21memory23:25:153
1475896992811,7cyclictest1498495-21python320:10:200
1475896992811,7cyclictest1498495-21python320:10:190
1475896992810,5cyclictest1538485-21cat22:05:200
15911262270,1sleep20-21swapper/200:35:182
1475900992718,7cyclictest1512252-21/usr/sbin/munin20:50:201
147589699279,5cyclictest1515318-21cat21:00:110
139099270,4rtkit-daemon1389-21rtkit-daemon00:34:560
15563752260,4chrt3056-21PK-Backend22:55:241
14981252260,2sleep20-21swapper/220:10:162
14981252260,2sleep20-21swapper/220:10:162
1475907992614,11cyclictest42-21ksoftirqd/323:20:003
147589699260,25cyclictest0-21swapper/020:15:590
139099260,4rtkit-daemon1389-21rtkit-daemon20:46:420
147589699259,3cyclictest1503415-21missed_timers20:25:180
147589699258,5cyclictest1564481-21expr23:20:130
147589699258,5cyclictest1489457-21ntpq19:45:200
147589699258,4cyclictest1527737-21sed21:35:180
1475896992512,3cyclictest1494362-21cstates20:00:120
139099250,4rtkit-daemon1389-21rtkit-daemon21:25:341
1475900992417,5cyclictest1492170-21latency_hist19:55:011
147589699249,3cyclictest2019-21gmain19:40:000
147589699249,3cyclictest1540458-21timerandwakeup22:10:220
147590399230,22cyclictest0-21swapper/220:40:312
1475900992321,1cyclictest0-21swapper/121:55:511
147589699239,2cyclictest1493899-21latency_hist20:00:000
1475896992321,1cyclictest0-21swapper/021:49:140
1475896992321,1cyclictest0-21swapper/019:17:320
147589699232,10cyclictest1482664-21grep19:25:220
147590799224,16cyclictest436-20systemd-journal23:40:013
1475907992216,4cyclictest1514697-21sh21:00:013
1475907992214,6cyclictest1483247-21/usr/sbin/munin19:30:003
1475907992213,7cyclictest1554702-21users22:50:193
1475903992220,1cyclictest0-21swapper/222:44:562
1475903992220,1cyclictest0-21swapper/222:01:212
1475903992220,1cyclictest0-21swapper/221:58:252
1475903992220,1cyclictest0-21swapper/221:20:192
1475903992220,1cyclictest0-21swapper/220:50:392
1475903992220,1cyclictest0-21swapper/200:09:522
1475903992219,2cyclictest3056-21PK-Backend00:15:052
1475903992218,2cyclictest34-21ksoftirqd/221:09:592
1475903992215,5cyclictest34-21ksoftirqd/219:50:122
147590399221,20cyclictest886-21NetworkManager19:20:012
147590399220,21cyclictest0-21swapper/221:28:032
147590099225,14cyclictest1486699-21cron19:40:001
1475900992220,1cyclictest3056-21PK-Backend23:56:221
1475900992220,1cyclictest3056-21PK-Backend23:56:221
1475900992220,1cyclictest0-21swapper/123:05:531
1475900992220,1cyclictest0-21swapper/121:34:211
1475900992220,1cyclictest0-21swapper/120:18:511
147590099220,3cyclictest1543539-21memory22:20:181
147590099220,20cyclictest3056-21PK-Backend22:25:111
147589699222,7cyclictest1518429-21apt21:10:110
1475896992220,1cyclictest0-21swapper/023:47:440
1475896992220,1cyclictest0-21swapper/022:21:310
1475896992220,1cyclictest0-21swapper/022:04:560
1475896992220,1cyclictest0-21swapper/019:40:100
1475896992220,1cyclictest0-21swapper/000:06:420
1475896992217,4cyclictest0-21swapper/021:32:170
1475896992210,3cyclictest1095-21runrttasks00:29:290
15420692210,4chrt3056-21PK-Backend22:15:203
147590799210,20cyclictest3056-21PK-Backend20:48:053
147590799210,1cyclictest3056-21PK-Backend00:16:463
147590799210,19cyclictest3056-21PK-Backend22:46:133
147590799210,19cyclictest3056-21PK-Backend22:12:083
147590799210,19cyclictest3056-21PK-Backend21:51:463
147590799210,19cyclictest3056-21PK-Backend21:25:563
147590799210,19cyclictest3056-21PK-Backend20:32:513
147590799210,19cyclictest3056-21PK-Backend20:25:213
147590799210,19cyclictest3056-21PK-Backend19:10:113
147590799210,19cyclictest3056-21PK-Backend00:28:223
1475903992119,1cyclictest0-21swapper/223:48:022
1475903992119,1cyclictest0-21swapper/223:37:142
1475903992119,1cyclictest0-21swapper/223:23:082
1475903992119,1cyclictest0-21swapper/222:07:502
1475903992119,1cyclictest0-21swapper/221:54:062
1475903992119,1cyclictest0-21swapper/221:31:142
1475903992119,1cyclictest0-21swapper/220:45:102
1475903992119,1cyclictest0-21swapper/219:39:412
1475903992119,1cyclictest0-21swapper/219:30:142
1475903992119,1cyclictest0-21swapper/200:29:332
1475903992116,4cyclictest34-21ksoftirqd/220:35:142
147590399210,20cyclictest0-21swapper/223:43:022
147590399210,20cyclictest0-21swapper/221:18:442
147590399210,20cyclictest0-21swapper/219:10:212
147590399210,20cyclictest0-21swapper/200:00:402
147590399210,1cyclictest0-21swapper/200:33:452
147590099212,14cyclictest82332-21update-manager00:04:071
1475900992119,1cyclictest0-21swapper/122:16:121
1475900992119,1cyclictest0-21swapper/121:07:451
1475900992119,1cyclictest0-21swapper/120:31:011
1475900992119,1cyclictest0-21swapper/120:27:081
1475900992119,1cyclictest0-21swapper/119:57:081
1475900992119,1cyclictest0-21swapper/119:22:061
1475900992119,1cyclictest0-21swapper/119:18:551
1475900992112,7cyclictest1540943-21cron22:14:591
147590099210,20cyclictest0-21swapper/120:24:181
147590099210,20cyclictest0-21swapper/100:06:581
147590099210,19cyclictest3056-21PK-Backend20:44:191
1475896992119,1cyclictest0-21swapper/020:35:520
1475896992113,6cyclictest1490861-21expr19:50:120
147589699210,20cyclictest0-21swapper/022:35:040
147589699210,20cyclictest0-21swapper/021:59:480
147589699210,20cyclictest0-21swapper/020:44:450
147589699210,19cyclictest0-21swapper/020:50:130
147590799205,2cyclictest141rcu_preempt20:00:003
1475907992018,1cyclictest3056-21PK-Backend22:08:023
1475907992018,1cyclictest0-21swapper/323:58:203
1475907992018,1cyclictest0-21swapper/323:58:203
1475907992018,1cyclictest0-21swapper/323:08:113
1475907992017,2cyclictest3056-21PK-Backend22:03:093
1475907992017,2cyclictest3056-21PK-Backend21:35:583
1475907992017,2cyclictest3056-21PK-Backend21:03:533
1475907992017,2cyclictest3056-21PK-Backend00:20:583
1475907992017,2cyclictest3056-21PK-Backend00:03:203
1475907992013,5cyclictest1556420-21cat22:55:243
1475907992012,6cyclictest1575621-21threads23:50:223
1475907992012,6cyclictest1548528-21df22:35:133
1475907992012,6cyclictest1512219-21awk20:50:203
147590799200,6cyclictest1497370-21munin-plugin-st20:10:003
147590799200,1cyclictest3056-21PK-Backend19:43:303
147590799200,1cyclictest3056-21PK-Backend00:34:573
147590799200,19cyclictest3056-21PK-Backend21:07:253
147590799200,19cyclictest0-21swapper/322:27:213
147590799200,19cyclictest0-21swapper/320:41:303
147590799200,19cyclictest0-21swapper/319:36:273
147590799200,19cyclictest0-21swapper/319:33:023
147590799200,19cyclictest0-21swapper/300:13:203
147590799200,19cyclictest0-21swapper/300:09:553
147590799200,18cyclictest3056-21PK-Backend21:16:523
1475903992019,1cyclictest0-21swapper/219:43:192
1475903992018,1cyclictest0-21swapper/223:05:312
1475903992018,1cyclictest0-21swapper/222:28:132
1475903992018,1cyclictest0-21swapper/221:44:522
1475903992018,1cyclictest0-21swapper/220:00:152
1475903992018,1cyclictest0-21swapper/219:56:382
1475903992018,1cyclictest0-21swapper/200:21:552
1475903992018,1cyclictest0-21swapper/200:12:292
1475903992014,5cyclictest34-21ksoftirqd/222:25:002
1475903992014,4cyclictest2156-21gnome-shell23:25:082
1475903992012,6cyclictest34-21ksoftirqd/220:30:222
1475903992012,2cyclictest34-21ksoftirqd/223:00:132
147590399200,1cyclictest0-21swapper/223:10:032
147590399200,1cyclictest0-21swapper/222:58:332
147590399200,19cyclictest0-21swapper/222:45:432
147590399200,19cyclictest0-21swapper/222:39:012
147590399200,19cyclictest0-21swapper/222:32:312
147590399200,19cyclictest0-21swapper/221:14:102
147590399200,19cyclictest0-21swapper/220:18:482
1475900992018,1cyclictest3056-21PK-Backend23:40:211
1475900992018,1cyclictest3056-21PK-Backend23:29:531
1475900992018,1cyclictest0-21swapper/123:30:401
1475900992018,1cyclictest0-21swapper/122:47:151
1475900992018,1cyclictest0-21swapper/122:06:391
1475900992018,1cyclictest0-21swapper/122:00:101
1475900992018,1cyclictest0-21swapper/120:55:141
1475900992018,1cyclictest0-21swapper/120:47:371
1475900992018,1cyclictest0-21swapper/120:08:221
1475900992014,4cyclictest2156-21gnome-shell21:37:091
147590099201,13cyclictest1495035-21chrt20:00:211
147590099200,1cyclictest3056-21PK-Backend23:10:121
147590099200,1cyclictest3056-21PK-Backend19:34:031
147590099200,1cyclictest0-21swapper/121:00:421
147590099200,1cyclictest0-21swapper/100:36:321
147590099200,19cyclictest3056-21PK-Backend22:30:391
147590099200,19cyclictest0-21swapper/121:50:101
147590099200,19cyclictest0-21swapper/121:21:441
147590099200,19cyclictest0-21swapper/121:15:001
147590099200,19cyclictest0-21swapper/119:41:251
147590099200,19cyclictest0-21swapper/100:15:431
147589699201,9cyclictest1522643-21ntp_states21:20:200
1475896992018,1cyclictest0-21swapper/023:14:450
1475896992018,1cyclictest0-21swapper/022:56:330
1475896992018,1cyclictest0-21swapper/022:52:000
1475896992018,1cyclictest0-21swapper/021:28:310
1475896992018,1cyclictest0-21swapper/000:12:300
147589699200,2cyclictest1572214-21sed23:40:190
147589699200,1cyclictest0-21swapper/022:41:330
147589699200,19cyclictest0-21swapper/023:08:360
147589699200,19cyclictest0-21swapper/022:34:160
147589699200,19cyclictest0-21swapper/022:26:250
147589699200,19cyclictest0-21swapper/022:15:160
147589699200,19cyclictest0-21swapper/021:17:030
139099200,16rtkit-daemon3056-21PK-Backend19:20:200
1475907991917,1cyclictest3056-21PK-Backend23:20:273
1475907991917,1cyclictest3056-21PK-Backend19:47:083
1475907991917,1cyclictest0-21swapper/323:32:453
1475907991917,1cyclictest0-21swapper/323:02:513
1475907991916,2cyclictest3056-21PK-Backend20:21:423
1475907991916,2cyclictest3056-21PK-Backend20:01:483
1475907991914,3cyclictest978-21snmpd22:34:013
1475907991913,5cyclictest1561519-21sed23:10:213
1475907991912,5cyclictest1571447-21expr23:40:103
147590799190,1cyclictest3056-21PK-Backend21:40:043
147590799190,1cyclictest3056-21PK-Backend21:20:373
147590799190,1cyclictest3056-21PK-Backend20:13:383
147590799190,1cyclictest3056-21PK-Backend20:13:373
147590799190,18cyclictest3056-21PK-Backend22:20:213
147590799190,18cyclictest3056-21PK-Backend21:45:313
1475903991917,1cyclictest0-21swapper/223:17:002
1475903991917,1cyclictest0-21swapper/222:51:042
1475903991917,1cyclictest0-21swapper/220:06:312
1475903991917,1cyclictest0-21swapper/219:46:102
1475903991915,3cyclictest978-21snmpd19:29:532
1475903991912,5cyclictest1503277-21awk20:25:162
1475903991911,6cyclictest1501547-21cat20:20:152
147590399190,1cyclictest0-21swapper/221:38:372
1475900991917,1cyclictest3056-21PK-Backend22:40:111
1475900991917,1cyclictest3056-21PK-Backend20:35:541
1475900991917,1cyclictest0-21swapper/120:11:001
1475900991917,1cyclictest0-21swapper/120:11:001
1475900991917,1cyclictest0-21swapper/119:47:411
1475900991916,2cyclictest3056-21PK-Backend23:45:421
1475900991913,3cyclictest1563756-21latency_hist23:20:011
147590099190,1cyclictest3056-21PK-Backend23:00:131
147590099190,18cyclictest0-21swapper/123:21:491
147590099190,18cyclictest0-21swapper/119:11:311
147589699193,3cyclictest0-21swapper/000:00:200
1475896991917,1cyclictest0-21swapper/019:34:560
1475896991917,1cyclictest0-21swapper/000:20:080
1475896991914,4cyclictest1570948-21grep23:40:010
1475896991914,3cyclictest1591217-21grep00:35:180
147589699190,1cyclictest3056-21PK-Backend23:15:320
147589699190,18cyclictest0-21swapper/020:32:490
1475907991816,1cyclictest3056-21PK-Backend21:58:483
1475907991816,1cyclictest3056-21PK-Backend20:15:273
1475907991812,4cyclictest1480198-21grep19:20:123
1475907991811,5cyclictest1550376-21expr22:40:133
1475903991816,1cyclictest0-21swapper/223:56:272
1475903991816,1cyclictest0-21swapper/223:56:262
1475903991816,1cyclictest0-21swapper/223:53:022
1475903991816,1cyclictest0-21swapper/223:33:152
1475903991816,1cyclictest0-21swapper/221:47:032
1475903991811,2cyclictest34-21ksoftirqd/222:15:152
1475903991811,2cyclictest34-21ksoftirqd/219:20:312
147590399180,1cyclictest0-21swapper/220:55:322
147590099188,9cyclictest3056-21PK-Backend22:55:011
1475900991816,1cyclictest3056-21PK-Backend22:37:271
1475900991816,1cyclictest3056-21PK-Backend00:10:571
1475900991811,5cyclictest1529427-21sed21:40:171
147590099181,12cyclictest1530285-21apt-get21:45:031
1475896991816,1cyclictest0-21swapper/023:27:500
1475896991816,1cyclictest0-21swapper/020:23:160
1475896991812,5cyclictest1477684-21sed19:15:010
147590799178,8cyclictest1506450-21cpuspeed_turbos20:35:133
1475907991715,1cyclictest3056-21PK-Backend19:51:203
1475907991715,1cyclictest3056-21PK-Backend00:39:563
1475900991715,1cyclictest0-21swapper/100:24:561
1475900991713,3cyclictest1587891-21proc_pri00:25:221
147589699179,6cyclictest436-20systemd-journal23:55:000
1475896991710,5cyclictest1517327-21cat21:05:170
1475907991611,4cyclictest42-21ksoftirqd/321:34:133
147590099169,5cyclictest1518630-21cat21:10:121
147590099168,6cyclictest1589544-21ntp_kernel_pll_00:30:191
1475896991611,3cyclictest1529494-21needreboot21:40:170
1475896991610,4cyclictest1514707-21turbostat21:00:000
147590799159,5cyclictest42-21ksoftirqd/319:20:003
1475903991513,1cyclictest0-21swapper/222:10:012
1475900991513,1cyclictest3056-21PK-Backend23:50:151
1475900991412,1cyclictest3056-21PK-Backend23:39:321
1475896991412,1cyclictest0-21swapper/023:33:100
147590099139,3cyclictest3056-21PK-Backend19:25:401
1475896991311,1cyclictest0-21swapper/023:59:200
1475896991311,1cyclictest0-21swapper/023:59:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional