You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-17 - 07:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Sun May 17, 2026 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6496121050,3sleep00-21swapper/019:40:190
904612900,1sleep00-21swapper/020:55:020
1661462810,12sleep126-21ksoftirqd/100:30:121
5292228043,14sleep30-21swapper/319:09:073
5293027940,13sleep10-21swapper/119:09:131
5265127239,9sleep00-21swapper/019:05:380
5282526242,14sleep20-21swapper/219:07:542
1419632530,4sleep13056-21PK-Backend23:20:151
663152400,4sleep00-21swapper/019:45:120
139099400,4rtkit-daemon1389-21rtkit-daemon20:24:200
139099400,4rtkit-daemon1389-21rtkit-daemon20:24:190
1509392350,2sleep20-21swapper/223:45:182
908722320,4sleep1261ksoftirqd/120:55:151
1242552320,1sleep3124256-21expr22:30:123
5312299270,25cyclictest1389-21rtkit-daemon20:16:370
780852250,8sleep234-21ksoftirqd/220:20:112
780852250,8sleep234-21ksoftirqd/220:20:112
53129992417,5cyclictest42-21ksoftirqd/323:24:043
5312999241,3cyclictest105629-21ntp_states21:35:193
53126992419,4cyclictest34-21ksoftirqd/200:39:072
139099240,16rtkit-daemon0-21swapper/220:39:022
53126992316,5cyclictest160666-21cron00:15:002
5312499230,21cyclictest93703-21apt-get21:05:041
5312299237,3cyclictest1412652date23:20:000
53122992316,5cyclictest93703-21apt-get21:05:000
5312999221,9cyclictest154420-21/usr/sbin/munin23:55:193
53126992220,1cyclictest0-21swapper/200:18:152
5312699220,21cyclictest0-21swapper/200:23:022
53124992220,1cyclictest3056-21PK-Backend19:13:371
53124992220,1cyclictest0-21swapper/122:15:411
53124992220,1cyclictest0-21swapper/122:15:411
53124992220,1cyclictest0-21swapper/121:01:391
53124992220,1cyclictest0-21swapper/120:49:151
53124992220,1cyclictest0-21swapper/100:25:401
53124992219,2cyclictest3056-21PK-Backend23:33:281
53124992219,2cyclictest3056-21PK-Backend00:13:501
53124992214,6cyclictest883-21dbus-daemon20:10:011
53124992213,7cyclictest163073-21cat00:20:131
5312499220,20cyclictest3056-21PK-Backend21:55:471
5312299220,20cyclictest3056-21PK-Backend21:30:230
5312299220,20cyclictest3056-21PK-Backend21:25:230
960102210,3chrt0-21swapper/321:10:123
960102210,3chrt0-21swapper/321:10:123
561262210,3sleep00-21swapper/019:15:170
53129992113,6cyclictest66080-21expr19:45:123
53129992113,6cyclictest139479-21sed23:15:003
5312999210,1cyclictest978-21snmpd21:20:193
5312999210,1cyclictest3056-21PK-Backend19:39:003
5312999210,19cyclictest3056-21PK-Backend19:27:383
5312999210,19cyclictest0-21swapper/319:40:013
53126992119,1cyclictest0-21swapper/223:23:132
53126992119,1cyclictest0-21swapper/223:18:532
53126992119,1cyclictest0-21swapper/223:04:412
53126992119,1cyclictest0-21swapper/222:13:582
53126992119,1cyclictest0-21swapper/221:37:002
53126992119,1cyclictest0-21swapper/220:06:022
53126992119,1cyclictest0-21swapper/220:02:382
53126992119,1cyclictest0-21swapper/219:56:492
53126992119,1cyclictest0-21swapper/219:51:562
53126992119,1cyclictest0-21swapper/219:19:452
53126992113,6cyclictest883-21dbus-daemon22:39:032
5312699210,20cyclictest59025-21apt19:25:122
5312699210,20cyclictest0-21swapper/223:50:372
5312699210,20cyclictest0-21swapper/222:32:432
5312699210,20cyclictest0-21swapper/222:18:512
5312699210,20cyclictest0-21swapper/222:18:512
5312699210,20cyclictest0-21swapper/222:06:142
5312699210,20cyclictest0-21swapper/221:53:362
5312699210,20cyclictest0-21swapper/221:46:062
5312699210,20cyclictest0-21swapper/219:30:532
5312699210,20cyclictest0-21swapper/200:07:132
5312699210,20cyclictest0-21swapper/200:04:432
53124992119,1cyclictest3056-21PK-Backend23:29:561
53124992119,1cyclictest3056-21PK-Backend22:26:151
53124992119,1cyclictest3056-21PK-Backend00:08:021
53124992119,1cyclictest0-21swapper/122:51:171
53124992119,1cyclictest0-21swapper/122:32:581
53124992119,1cyclictest0-21swapper/120:33:071
53124992119,1cyclictest0-21swapper/119:39:401
53124992119,1cyclictest0-21swapper/119:26:281
53124992118,2cyclictest3056-21PK-Backend23:09:351
53124992118,2cyclictest3056-21PK-Backend22:35:011
53124992115,4cyclictest57951-21needreboot19:20:181
53124992115,4cyclictest57951-21needreboot19:20:171
5312499210,20cyclictest3056-21PK-Backend20:14:011
5312499210,20cyclictest0-21swapper/120:26:171
5312499210,19cyclictest3056-21PK-Backend22:02:161
5312499210,19cyclictest3056-21PK-Backend19:45:351
5312499210,19cyclictest3056-21PK-Backend19:44:331
5312499210,19cyclictest3056-21PK-Backend00:15:051
53122992119,1cyclictest0-21swapper/021:51:460
53122992119,1cyclictest0-21swapper/019:10:310
53122992115,4cyclictest167606-21cron00:35:000
53122992114,5cyclictest57991-21ntp_kernel_err19:20:180
53122992114,5cyclictest57991-21ntp_kernel_err19:20:180
53122992110,10cyclictest155420-21cat00:00:010
5312299210,20cyclictest0-21swapper/021:56:260
5312299210,20cyclictest0-21swapper/000:37:470
53129992018,1cyclictest0-21swapper/323:52:483
53129992018,1cyclictest0-21swapper/321:26:073
53129992017,2cyclictest3056-21PK-Backend20:06:443
53129992015,3cyclictest159955-21ntp_states00:10:193
53129992015,3cyclictest156143-21hddtemp_smartct00:00:133
53129992013,5cyclictest139728-21expr23:15:143
53129992012,6cyclictest883-21dbus-daemon00:10:013
53129992012,6cyclictest53780-21cat19:10:173
5312999200,1cyclictest0-21swapper/322:22:583
5312999200,19cyclictest3056-21PK-Backend21:48:383
5312999200,19cyclictest3056-21PK-Backend20:01:443
5312999200,19cyclictest3056-21PK-Backend00:18:583
5312999200,19cyclictest0-21swapper/322:57:263
5312999200,19cyclictest0-21swapper/321:31:073
5312999200,19cyclictest0-21swapper/321:09:583
5312999200,18cyclictest126573-21python322:35:213
5312999200,17cyclictest127431-21/usr/sbin/munin22:40:153
53126992018,1cyclictest0-21swapper/223:06:572
53126992018,1cyclictest0-21swapper/222:52:312
53126992018,1cyclictest0-21swapper/222:43:322
53126992018,1cyclictest0-21swapper/220:57:122
53126992018,1cyclictest0-21swapper/219:21:342
53126992018,1cyclictest0-21swapper/219:21:342
53126992018,1cyclictest0-21swapper/200:33:362
53126992018,1cyclictest0-21swapper/200:27:482
53126992017,2cyclictest0-21swapper/221:15:242
53126992017,2cyclictest0-21swapper/221:15:242
53126992012,6cyclictest113317-21munin-plugin-st22:00:002
5312699200,19cyclictest0-21swapper/223:26:312
5312699200,19cyclictest0-21swapper/223:10:152
5312699200,19cyclictest0-21swapper/222:47:242
5312699200,19cyclictest0-21swapper/222:20:542
5312699200,19cyclictest0-21swapper/219:40:402
5312699200,19cyclictest0-21swapper/219:38:382
53124992018,1cyclictest3056-21PK-Backend23:46:191
53124992018,1cyclictest3056-21PK-Backend21:24:581
53124992018,1cyclictest0-21swapper/122:40:421
53124992018,1cyclictest0-21swapper/121:48:171
53124992018,1cyclictest0-21swapper/120:16:371
53124992017,2cyclictest3056-21PK-Backend23:41:461
53124992013,5cyclictest70782-21sshd19:58:551
5312499200,1cyclictest0-21swapper/120:35:301
5312499200,19cyclictest3056-21PK-Backend23:56:261
5312499200,19cyclictest3056-21PK-Backend23:16:591
5312499200,19cyclictest3056-21PK-Backend22:57:521
5312499200,19cyclictest3056-21PK-Backend22:22:231
5312499200,19cyclictest3056-21PK-Backend00:00:591
5312499200,19cyclictest0-21swapper/122:11:491
5312499200,19cyclictest0-21swapper/122:07:571
5312499200,19cyclictest0-21swapper/120:21:511
5312499200,19cyclictest0-21swapper/120:21:511
5312499200,19cyclictest0-21swapper/120:00:351
5312499200,19cyclictest0-21swapper/119:18:241
53122992018,1cyclictest0-21swapper/023:40:080
53122992018,1cyclictest0-21swapper/023:13:520
53122992018,1cyclictest0-21swapper/022:05:110
53122992018,1cyclictest0-21swapper/000:16:520
53122992018,1cyclictest0-21swapper/000:08:200
53122992013,5cyclictest883-21dbus-daemon20:50:010
53122992012,6cyclictest131993-21users22:50:230
53122992010,8cyclictest883-21dbus-daemon20:04:000
5312299200,1cyclictest0-21swapper/021:18:480
5312299200,1cyclictest0-21swapper/021:18:470
5312299200,19cyclictest0-21swapper/022:48:440
5312299200,19cyclictest0-21swapper/022:43:030
5312299200,19cyclictest0-21swapper/022:19:370
5312299200,19cyclictest0-21swapper/022:19:370
5312299200,19cyclictest0-21swapper/022:11:540
5312299200,19cyclictest0-21swapper/022:00:180
5312299200,19cyclictest0-21swapper/021:10:500
5312299200,19cyclictest0-21swapper/021:10:500
5312299200,19cyclictest0-21swapper/021:06:510
5312299200,19cyclictest0-21swapper/020:27:510
5312299200,19cyclictest0-21swapper/019:37:550
5312299200,19cyclictest0-21swapper/019:33:020
139099200,13rtkit-daemon3056-21PK-Backend22:12:163
53129991917,1cyclictest3056-21PK-Backend19:17:573
53129991917,1cyclictest0-21swapper/323:27:543
53129991917,1cyclictest0-21swapper/320:25:373
53129991917,1cyclictest0-21swapper/320:11:453
53129991917,1cyclictest0-21swapper/319:30:283
53129991917,1cyclictest0-21swapper/300:20:133
53129991914,3cyclictest98722-21open_inodes21:15:223
53129991914,3cyclictest98722-21open_inodes21:15:213
53129991914,3cyclictest119169-21hddtemp_smartct22:15:153
53129991914,3cyclictest119169-21hddtemp_smartct22:15:153
5312999191,16cyclictest76360-21apt20:15:113
5312999191,16cyclictest147653-21python323:35:213
5312999190,7cyclictest135460-21users23:00:233
5312999190,1cyclictest3056-21PK-Backend22:47:593
5312999190,1cyclictest3056-21PK-Backend22:08:323
5312999190,1cyclictest3056-21PK-Backend20:50:113
5312999190,1cyclictest3056-21PK-Backend19:54:073
5312999190,1cyclictest0-21swapper/320:55:113
5312999190,18cyclictest3056-21PK-Backend21:04:243
5312999190,18cyclictest3056-21PK-Backend20:31:393
5312999190,18cyclictest0-21swapper/320:41:593
5312999190,18cyclictest0-21swapper/319:22:103
5312999190,18cyclictest0-21swapper/319:22:103
5312999190,18cyclictest0-21swapper/300:38:383
5312699192,15cyclictest101147-21sh21:25:002
53126991917,1cyclictest0-21swapper/223:40:032
53126991917,1cyclictest0-21swapper/223:31:312
53126991917,1cyclictest0-21swapper/220:49:352
53126991917,1cyclictest0-21swapper/220:17:172
53126991917,1cyclictest0-21swapper/219:11:002
53126991915,3cyclictest978-21snmpd20:51:572
5312699190,1cyclictest3056-21PK-Backend21:40:252
5312699190,18cyclictest0-21swapper/222:01:202
53124991917,1cyclictest3056-21PK-Backend23:14:561
53124991917,1cyclictest0-21swapper/121:52:561
53124991917,1cyclictest0-21swapper/121:10:381
53124991917,1cyclictest0-21swapper/121:10:381
53124991917,1cyclictest0-21swapper/100:36:001
53124991914,3cyclictest108101-21sh21:45:001
5312499190,1cyclictest3056-21PK-Backend21:30:471
5312499190,1cyclictest0-21swapper/121:37:221
5312499190,18cyclictest0-21swapper/119:31:081
5312299199,8cyclictest883-21dbus-daemon21:39:020
53122991917,1cyclictest3056-21PK-Backend23:50:430
53122991917,1cyclictest0-21swapper/023:47:110
53122991917,1cyclictest0-21swapper/023:23:320
53122991917,1cyclictest0-21swapper/023:05:140
53122991917,1cyclictest0-21swapper/021:48:550
53122991917,1cyclictest0-21swapper/020:44:130
53122991917,1cyclictest0-21swapper/000:14:220
53122991914,4cyclictest13-21ksoftirqd/020:55:020
53122991912,5cyclictest143477-21tr23:25:130
5312299190,1cyclictest0-21swapper/023:32:510
5312299190,1cyclictest0-21swapper/022:34:180
5312299190,1cyclictest0-21swapper/022:26:340
5312299190,1cyclictest0-21swapper/022:24:310
5312299190,1cyclictest0-21swapper/020:11:550
5312299190,18cyclictest0-21swapper/020:09:180
53129991816,1cyclictest0-21swapper/322:02:513
53129991816,1cyclictest0-21swapper/300:26:013
5312999181,15cyclictest150734-21irqstats23:45:163
5312999181,13cyclictest145059-21apt23:30:123
53129991810,6cyclictest112305-21/usr/sbin/munin21:55:173
5312999180,17cyclictest3056-21PK-Backend20:23:493
5312999180,17cyclictest3056-21PK-Backend20:23:483
5312999180,16cyclictest84803-21munin-run20:40:003
5312699189,7cyclictest2254-21gdbus23:59:052
53126991816,1cyclictest0-21swapper/220:30:502
53126991816,1cyclictest0-21swapper/220:27:112
53126991816,1cyclictest0-21swapper/219:45:472
53126991813,3cyclictest85985-21timerandwakeup20:40:212
53126991812,4cyclictest102912-21systemctl21:30:002
53124991815,2cyclictest3056-21PK-Backend23:35:581
53124991815,2cyclictest3056-21PK-Backend23:02:121
53124991813,3cyclictest129396-21cstates22:45:111
53124991811,5cyclictest883-21dbus-daemon21:30:001
53124991810,6cyclictest98312-21df21:15:151
53124991810,6cyclictest98312-21df21:15:151
5312499180,17cyclictest3056-21PK-Backend23:51:521
53122991816,1cyclictest0-21swapper/022:56:210
53122991816,1cyclictest0-21swapper/021:41:260
53122991814,3cyclictest135852-21turbostat.cron23:05:000
53122991811,5cyclictest83258-21cut20:35:010
53122991810,6cyclictest162689-21apt00:20:110
53129991715,1cyclictest3056-21PK-Backend21:51:283
53129991715,1cyclictest3056-21PK-Backend21:41:213
53129991715,1cyclictest0-21swapper/322:54:083
53129991715,1cyclictest0-21swapper/322:27:523
53129991710,5cyclictest166665-21cut00:30:183
5312999170,16cyclictest0-21swapper/323:08:223
53126991715,1cyclictest0-21swapper/223:38:472
53126991715,1cyclictest0-21swapper/221:33:352
53126991715,1cyclictest0-21swapper/221:04:012
5312499179,5cyclictest88687-21cpuspeed_turbos20:50:111
53122991715,1cyclictest0-21swapper/023:38:530
53122991715,1cyclictest0-21swapper/000:25:100
53122991715,1cyclictest0-21swapper/000:02:180
53122991712,3cyclictest60056-21timerwakeupswit19:25:230
53122991710,5cyclictest71074-21date20:00:010
5312999165,9cyclictest86963-21tr20:45:123
53129991611,3cyclictest148499-21/usr/sbin/munin23:40:163
53126991614,1cyclictest0-21swapper/222:57:032
53126991614,1cyclictest0-21swapper/222:28:512
53126991610,4cyclictest2156-21gnome-shell21:08:092
5312499169,5cyclictest68608-21cat19:50:211
5312499168,6cyclictest85194-21grep20:40:131
53122991610,5cyclictest83534-21expr20:35:110
53122991610,4cyclictest68186-21cut19:50:180
53122991610,4cyclictest2156-21gnome-shell22:39:090
53129991513,1cyclictest3056-21PK-Backend19:56:103
53126991511,2cyclictest34-21ksoftirqd/220:10:092
5312699140,13cyclictest34-21ksoftirqd/221:10:142
5312699140,13cyclictest34-21ksoftirqd/221:10:142
5312299149,4cyclictest101139-21cron21:25:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional