You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 16:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sat Aug 29, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139099850,72rtkit-daemon0-21swapper/307:09:193
5133662740,4sleep23056-21PK-Backend12:25:252
39860727448,16sleep10-21swapper/107:05:131
4350942640,2sleep00-21swapper/008:45:170
39996926243,13sleep00-21swapper/007:09:020
39989926140,14sleep20-21swapper/207:08:072
5144802540,3sleep00-21swapper/012:30:170
4602812500,4sleep342-21ksoftirqd/309:55:223
4488412450,2sleep10-21swapper/109:25:121
4989522440,2sleep30-21swapper/311:45:213
4510422430,3sleep00-21swapper/009:30:120
40022299395,19cyclictest0-21swapper/010:47:420
400232993215,15cyclictest446752-21cron09:20:003
40022299304,8cyclictest446913-21cat09:20:000
40023299290,17cyclictest0-21swapper/309:28:593
40022299264,9cyclictest409247-21taskset07:34:500
40022299264,8cyclictest426857-21taskset08:24:240
40022299263,8cyclictest439469-21taskset08:55:300
5161682250,11sleep342-21ksoftirqd/312:35:173
40023299250,24cyclictest0-21swapper/308:57:043
40023299250,23cyclictest425546-21ls08:20:123
40022299254,9cyclictest406083-21apt07:25:130
40022299254,8cyclictest0-21swapper/012:28:000
40022299253,8cyclictest414432-21taskset07:45:330
139099250,4rtkit-daemon1389-21rtkit-daemon12:38:240
139099250,15rtkit-daemon331rcuc/209:21:152
40023299249,13cyclictest436-20systemd-journal10:45:003
40023299249,13cyclictest0-21swapper/308:30:013
40023299240,23cyclictest0-21swapper/307:50:043
40022299247,7cyclictest433020-21df_inode08:40:150
40022299244,8cyclictest447243-21/usr/sbin/munin09:20:130
40022299244,7cyclictest0-21swapper/012:20:300
400222992418,4cyclictest453078-21memory09:35:180
40023299238,13cyclictest422798-21cut08:10:203
40023299238,13cyclictest1-21systemd07:35:013
400232992318,4cyclictest0-21swapper/312:23:133
400232992315,6cyclictest502333-21awk11:55:193
400232992315,6cyclictest2330-21gdbus11:26:043
40023299231,11cyclictest510013-21cron12:20:003
400232992311,11cyclictest506788-21/usr/sbin/munin12:10:163
40023299230,1cyclictest0-21swapper/309:24:003
400229992320,2cyclictest3056-21PK-Backend09:00:362
400227992317,4cyclictest436-20systemd-journal09:59:551
400227992315,6cyclictest26-21ksoftirqd/107:15:131
40022799231,20cyclictest474784-21/usr/sbin/munin10:40:001
40022799231,20cyclictest459033-21runrttasks09:53:371
40022299235,10cyclictest1095-21runrttasks10:30:260
40022299234,11cyclictest0-21swapper/009:25:260
400222992316,5cyclictest436-20systemd-journal08:10:000
40023299229,12cyclictest466332-21/usr/sbin/munin10:15:123
40023299229,11cyclictest468932-21processes10:20:213
40023299228,13cyclictest437423-21timerwakeupswit08:50:233
40023299228,12cyclictest474270-21timerandwakeup10:35:243
40023299228,12cyclictest457679-21acpi09:50:123
40023299227,13cyclictest442518-21netstat09:05:183
40023299223,17cyclictest495147-21if_err_enp2s011:35:173
40023299221,20cyclictest1-21systemd10:05:013
400232992210,10cyclictest473052-21unin-run10:35:003
400232992210,10cyclictest432381-21idleruntime-cro08:40:003
40023299220,13cyclictest413823-21kernelversion07:45:163
400229992220,1cyclictest3056-21PK-Backend10:26:202
400229992220,1cyclictest3056-21PK-Backend10:02:272
40022999220,20cyclictest3056-21PK-Backend11:40:152
40022999220,20cyclictest3056-21PK-Backend07:34:102
400227992220,1cyclictest0-21swapper/111:14:141
400227992220,1cyclictest0-21swapper/109:37:411
40022799220,21cyclictest0-21swapper/109:44:041
40022799220,21cyclictest0-21swapper/108:43:211
40022799220,10cyclictest4989552processes11:45:201
40022299224,6cyclictest480220-21taskset10:54:290
40022299223,6cyclictest455392-21taskset09:41:180
400222992215,3cyclictest404354-21apt07:20:130
400222992213,5cyclictest506814-21apt12:10:120
139099220,14rtkit-daemon3056-21PK-Backend08:06:022
4927912210,7sleep3411rcuc/311:30:133
40023299218,11cyclictest487211-21latency_hist11:15:003
40023299218,11cyclictest42-21ksoftirqd/307:30:003
400232992119,1cyclictest0-21swapper/312:03:463
400232992119,1cyclictest0-21swapper/310:28:493
400232992119,1cyclictest0-21swapper/309:45:563
400232992119,1cyclictest0-21swapper/308:02:143
40023299211,18cyclictest465964-21chrt10:13:383
40023299211,18cyclictest451424-21needreboot09:30:173
40023299211,18cyclictest440683-21memory09:00:183
40023299211,18cyclictest425185-21perf08:19:593
40023299210,20cyclictest0-21swapper/312:34:083
40023299210,20cyclictest0-21swapper/311:43:313
40023299210,20cyclictest0-21swapper/310:57:223
40023299210,20cyclictest0-21swapper/307:12:253
40023299210,19cyclictest402977-21df07:15:133
400229992119,1cyclictest3056-21PK-Backend11:32:312
400229992119,1cyclictest3056-21PK-Backend11:10:412
400229992119,1cyclictest3056-21PK-Backend10:54:122
400229992119,1cyclictest3056-21PK-Backend09:06:112
400229992119,1cyclictest3056-21PK-Backend07:23:362
400229992118,2cyclictest3056-21PK-Backend12:39:372
400229992118,2cyclictest3056-21PK-Backend10:58:582
400229992118,2cyclictest3056-21PK-Backend08:51:502
40022999210,19cyclictest3056-21PK-Backend12:00:022
40022999210,19cyclictest3056-21PK-Backend11:55:292
40022999210,19cyclictest3056-21PK-Backend10:32:432
40022799218,11cyclictest476515-21unin-run10:45:001
400227992119,1cyclictest0-21swapper/112:33:571
400227992119,1cyclictest0-21swapper/112:18:291
400227992119,1cyclictest0-21swapper/111:02:311
400227992119,1cyclictest0-21swapper/111:02:311
400227992119,1cyclictest0-21swapper/110:24:391
400227992119,1cyclictest0-21swapper/107:49:131
400227992112,7cyclictest436-20systemd-journal08:25:001
40022799210,2cyclictest493994-21python311:33:071
40022799210,20cyclictest0-21swapper/111:43:141
40022799210,20cyclictest0-21swapper/111:15:021
40022799210,20cyclictest0-21swapper/108:51:181
40022799210,20cyclictest0-21swapper/107:59:331
400222992113,6cyclictest467094-21ntp_states10:15:200
400222992110,9cyclictest510022-21latency_hist12:20:000
139099210,4rtkit-daemon1389-21rtkit-daemon12:00:441
139099210,14rtkit-daemon0-21swapper/308:09:533
40023299209,10cyclictest978-21snmpd09:10:393
40023299209,10cyclictest978-21snmpd07:35:123
40023299209,10cyclictest432795-21cpuspeed_turbos08:40:143
40023299203,15cyclictest0-21swapper/312:08:023
400232992018,1cyclictest0-21swapper/310:54:583
400232992015,4cyclictest0-21swapper/310:06:523
40023299200,19cyclictest0-21swapper/309:35:223
40023299200,19cyclictest0-21swapper/307:44:023
40023299200,18cyclictest430417-21sh08:35:003
400229992018,1cyclictest3056-21PK-Backend12:33:282
400229992018,1cyclictest3056-21PK-Backend11:54:422
400229992018,1cyclictest3056-21PK-Backend11:45:362
400229992018,1cyclictest3056-21PK-Backend11:39:272
400229992018,1cyclictest3056-21PK-Backend09:41:112
400229992018,1cyclictest3056-21PK-Backend09:35:382
400229992018,1cyclictest3056-21PK-Backend09:18:542
400229992017,2cyclictest3056-21PK-Backend08:01:212
40022999200,1cyclictest3056-21PK-Backend07:39:112
40022999200,19cyclictest3056-21PK-Backend11:29:202
40022999200,19cyclictest3056-21PK-Backend11:01:232
40022999200,19cyclictest3056-21PK-Backend11:01:222
40022999200,19cyclictest3056-21PK-Backend09:59:162
40022999200,19cyclictest3056-21PK-Backend09:51:322
40022999200,19cyclictest3056-21PK-Backend09:45:102
40022999200,18cyclictest3056-21PK-Backend11:08:182
400227992018,1cyclictest3056-21PK-Backend09:32:071
400227992018,1cyclictest0-21swapper/112:27:011
400227992018,1cyclictest0-21swapper/111:57:061
400227992018,1cyclictest0-21swapper/111:38:411
400227992018,1cyclictest0-21swapper/110:55:221
400227992018,1cyclictest0-21swapper/109:23:221
400227992018,1cyclictest0-21swapper/108:48:081
400227992018,1cyclictest0-21swapper/108:32:321
400227992018,1cyclictest0-21swapper/108:19:081
400227992013,5cyclictest431490-21cat08:35:181
40022799200,1cyclictest0-21swapper/112:38:301
40022799200,19cyclictest0-21swapper/112:20:051
40022799200,19cyclictest0-21swapper/111:50:101
40022799200,19cyclictest0-21swapper/110:27:501
40022799200,19cyclictest0-21swapper/109:19:241
40022799200,19cyclictest0-21swapper/109:10:381
40022799200,19cyclictest0-21swapper/108:10:091
40022799200,19cyclictest0-21swapper/107:52:231
40022799200,19cyclictest0-21swapper/107:14:251
40022299209,8cyclictest418530-21apt08:00:120
40022299207,12cyclictest1095-21runrttasks11:22:160
40022299206,12cyclictest1095-21runrttasks08:50:150
40022299202,16cyclictest1095-21runrttasks09:06:220
400222992013,5cyclictest0-21swapper/011:20:010
400222992012,5cyclictest430379-21apt-get08:35:030
400222992011,6cyclictest497282-21python311:40:210
400222992010,8cyclictest470015-21cut10:25:130
40022299200,2cyclictest474998-21/usr/sbin/munin10:40:140
40022299200,19cyclictest1389-21rtkit-daemon11:57:060
40022299200,18cyclictest1389-21rtkit-daemon11:08:400
40023299198,9cyclictest482257-21/usr/sbin/munin11:00:193
40023299198,9cyclictest482257-21/usr/sbin/munin11:00:193
40023299197,10cyclictest978-21snmpd07:22:323
40023299193,14cyclictest0-21swapper/308:49:573
40023299192,15cyclictest417658-21python307:55:223
40023299191,16cyclictest455703-21perf09:45:003
40023299190,18cyclictest0-21swapper/311:51:533
40023299190,17cyclictest512941-21tr12:25:213
40023299190,17cyclictest490282-21awk11:20:223
40023299190,17cyclictest485103-21chrt11:05:273
40023299190,17cyclictest478059-21apt-get10:47:393
40023299190,17cyclictest1788620-21WebKitWebProces11:19:553
400229991917,1cyclictest3056-21PK-Backend11:22:242
400229991917,1cyclictest3056-21PK-Backend08:19:052
400229991916,2cyclictest3056-21PK-Backend08:38:192
400229991910,7cyclictest883-21dbus-daemon09:30:002
40022999190,1cyclictest3056-21PK-Backend12:12:062
40022999190,1cyclictest3056-21PK-Backend11:18:252
40022999190,1cyclictest3056-21PK-Backend10:49:252
40022999190,1cyclictest3056-21PK-Backend10:37:292
40022999190,1cyclictest3056-21PK-Backend09:14:082
40022999190,1cyclictest3056-21PK-Backend08:47:042
40022999190,1cyclictest3056-21PK-Backend08:25:412
40022999190,1cyclictest3056-21PK-Backend07:45:122
40022999190,18cyclictest3056-21PK-Backend08:55:012
40022999190,18cyclictest3056-21PK-Backend07:18:492
400227991917,1cyclictest0-21swapper/108:28:191
400227991917,1cyclictest0-21swapper/107:29:461
400227991912,5cyclictest436-20systemd-journal11:26:041
40022799190,18cyclictest0-21swapper/112:07:001
40022299199,8cyclictest491733-21ps11:25:200
40022299199,8cyclictest482007-21sh11:00:000
40022299199,8cyclictest468301-21/usr/sbin/munin10:20:150
40022299199,8cyclictest458559-21python309:50:220
40022299199,8cyclictest441582-21turbostat09:05:000
40022299199,8cyclictest423922-21cstates08:15:120
40022299199,8cyclictest1095-21runrttasks07:50:230
40022299199,8cyclictest0-21swapper/009:45:110
40022299193,15cyclictest1095-21runrttasks09:11:430
400222991914,4cyclictest494873-21cpuspeed_turbos11:35:130
400222991913,3cyclictest498154-21apt11:45:120
400222991912,5cyclictest494192-21munin-run11:35:000
400222991911,6cyclictest505449-21df12:05:130
400222991911,6cyclictest461828-21needreboot10:00:180
400222991911,6cyclictest402774-21tr07:15:120
400222991911,5cyclictest499887-21apt11:50:110
400222991910,7cyclictest427992-21wc08:25:210
40022299190,1cyclictest0-21swapper/007:35:060
400229991816,1cyclictest3056-21PK-Backend10:05:392
400229991815,2cyclictest3056-21PK-Backend12:18:142
400229991815,2cyclictest3056-21PK-Backend08:40:552
400229991815,2cyclictest0-21swapper/209:30:032
40022999180,1cyclictest3056-21PK-Backend12:22:002
40022799189,7cyclictest883-21dbus-daemon10:10:001
400227991816,1cyclictest0-21swapper/110:15:061
400227991816,1cyclictest0-21swapper/107:38:111
400227991813,4cyclictest2156-21gnome-shell10:03:091
400227991813,3cyclictest490051-21processes11:20:201
400227991811,6cyclictest464624-21apt10:10:111
400227991811,5cyclictest472080-21cat10:30:161
400227991810,6cyclictest477146-21fw_packets10:45:161
40022799180,16cyclictest507483-21grep12:10:171
40022299189,7cyclictest460218-21ps09:55:220
40022299189,7cyclictest411742-21grep07:40:130
40022299188,8cyclictest463369-21irqstats10:05:180
40022299188,8cyclictest423007-21expr08:10:220
400222991814,3cyclictest1095-21runrttasks10:37:490
400222991813,3cyclictest428928-21apt08:30:120
400222991811,5cyclictest465692-21awk10:10:210
400222991811,5cyclictest436-20systemd-journal08:00:010
400222991810,6cyclictest503470-21cpuspeed_turbos12:00:110
400222991810,6cyclictest485899-21grep11:10:140
400229991715,1cyclictest3056-21PK-Backend10:15:592
400229991715,1cyclictest3056-21PK-Backend08:34:552
400229991715,1cyclictest3056-21PK-Backend07:41:002
400229991714,2cyclictest3056-21PK-Backend10:20:462
400229991714,2cyclictest3056-21PK-Backend07:50:122
40022999170,6cyclictest3056-21PK-Backend08:23:102
400227991715,1cyclictest0-21swapper/107:40:471
400227991715,1cyclictest0-21swapper/107:34:461
400227991711,5cyclictest457479-21sed09:50:021
400227991711,5cyclictest438539-21expr08:55:131
40022299179,6cyclictest400848-21cat07:10:160
40022299178,7cyclictest482430-21tr11:00:130
40022299178,7cyclictest482430-21tr11:00:130
400229991614,1cyclictest3056-21PK-Backend12:05:232
400229991614,1cyclictest3056-21PK-Backend07:11:542
400227991611,3cyclictest405228-21swap07:20:241
40022799161,12cyclictest484011-21apt11:05:111
400227991610,5cyclictest420183-21wc08:05:011
400227991610,4cyclictest479588-21awk10:50:201
400229991513,1cyclictest3056-21PK-Backend10:10:252
40022799159,4cyclictest420751-21kernelversion08:05:171
40022799158,5cyclictest440290-21cut09:00:131
40022799156,7cyclictest442414-21memory09:05:171
40022999139,3cyclictest3056-21PK-Backend07:28:042
40022999138,4cyclictest3056-21PK-Backend08:13:552
400229991311,1cyclictest3056-21PK-Backend07:55:472
40022999117,3cyclictest3056-21PK-Backend10:42:322
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional