You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-20 - 04:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Sat Jun 20, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
252088121110,4sleep13056-21PK-Backend19:30:171
251768421070,4sleep03056-21PK-Backend19:20:330
262896621040,2sleep00-21swapper/000:35:180
139099840,72rtkit-daemon0-21swapper/119:05:351
251057227847,22sleep20-21swapper/219:05:112
251172026343,14sleep00-21swapper/019:06:060
251170326348,9sleep30-21swapper/319:05:523
25726132550,2sleep23056-21PK-Backend21:55:222
25576732550,4sleep33056-21PK-Backend21:15:133
25986742470,2sleep10-21swapper/123:10:141
25373292440,5sleep33056-21PK-Backend20:15:223
25212002440,1sleep30-21swapper/319:30:233
26024302390,2sleep00-21swapper/023:20:170
25313752390,2sleep13056-21PK-Backend20:00:121
2512256992915,9cyclictest42-21ksoftirqd/319:25:113
2512252992925,3cyclictest34-21ksoftirqd/220:24:092
2512252992915,12cyclictest2532696-21cron20:05:002
25997902270,4sleep23056-21PK-Backend23:15:012
2512252992716,4cyclictest34-21ksoftirqd/221:44:092
2512252992713,12cyclictest2585289-21unin-run22:35:002
251225299270,6cyclictest3056-21PK-Backend22:15:002
2512252992622,3cyclictest331rcuc/223:05:052
2512252992612,12cyclictest34-21ksoftirqd/220:20:012
251224299261,9cyclictest2555521-21perf21:10:010
139099250,4rtkit-daemon1389-21rtkit-daemon22:07:011
2512252992420,3cyclictest331rcuc/219:47:492
251224299240,22cyclictest2538845-21hddtemp_smartct20:20:160
2512256992317,4cyclictest2629647-21cron00:39:593
2512256992313,8cyclictest2597900-21cron23:10:013
2512252992319,3cyclictest331rcuc/223:27:302
2512252992319,3cyclictest331rcuc/222:45:412
2512252992319,3cyclictest331rcuc/222:41:402
2512252992319,3cyclictest331rcuc/219:10:442
2512252992319,3cyclictest331rcuc/200:25:492
2512252992319,3cyclictest331rcuc/200:11:222
2512252992318,4cyclictest34-21ksoftirqd/219:39:052
2512252992318,4cyclictest34-21ksoftirqd/219:39:052
2512252992317,4cyclictest883-21dbus-daemon22:10:002
251224299239,12cyclictest2573111-21cron21:59:590
2512242992312,9cyclictest1-21systemd21:05:000
26089532220,3sleep33056-21PK-Backend23:40:123
2512256992220,1cyclictest3056-21PK-Backend20:13:063
2512256992219,2cyclictest3056-21PK-Backend21:25:243
2512256992219,2cyclictest0-21swapper/320:44:223
2512256992214,6cyclictest2586387-21python322:35:193
251225699220,20cyclictest3056-21PK-Backend22:24:403
251225699220,20cyclictest3056-21PK-Backend19:22:503
251225299224,7cyclictest34-21ksoftirqd/220:45:112
2512252992220,1cyclictest0-21swapper/223:45:452
2512252992220,1cyclictest0-21swapper/221:04:512
2512252992220,1cyclictest0-21swapper/220:44:222
2512252992220,1cyclictest0-21swapper/220:25:512
2512252992219,2cyclictest331rcuc/222:54:552
2512252992218,3cyclictest331rcuc/223:56:142
2512252992218,3cyclictest331rcuc/223:51:562
2512252992218,3cyclictest331rcuc/223:21:012
2512252992218,3cyclictest331rcuc/223:18:052
2512252992218,3cyclictest331rcuc/223:01:522
2512252992218,3cyclictest331rcuc/220:33:242
2512252992218,3cyclictest331rcuc/220:12:062
2512252992218,3cyclictest331rcuc/219:58:042
2512252992218,3cyclictest331rcuc/219:40:022
2512252992218,3cyclictest331rcuc/219:34:062
2512252992218,3cyclictest331rcuc/219:16:482
2512252992218,3cyclictest331rcuc/200:35:282
2512252992218,3cyclictest331rcuc/200:05:452
2512252992217,4cyclictest34-21ksoftirqd/223:45:012
251225299220,9cyclictest25699041/usr/sbin/munin21:50:202
2512249992220,1cyclictest0-21swapper/119:54:511
2512249992219,2cyclictest3056-21PK-Backend23:25:561
2512249992214,6cyclictest436-20systemd-journal21:49:591
251224999220,21cyclictest0-21swapper/122:22:411
251224999220,20cyclictest3056-21PK-Backend23:58:191
2512242992220,1cyclictest0-21swapper/023:55:520
2512242992220,1cyclictest0-21swapper/023:42:120
2512242992220,1cyclictest0-21swapper/022:56:240
2512242992220,1cyclictest0-21swapper/021:49:450
2512242992220,1cyclictest0-21swapper/020:17:050
2512242992220,1cyclictest0-21swapper/019:48:460
2512242992218,3cyclictest2550310-21cron20:55:000
2512242992216,5cyclictest2525509-21apt19:45:000
251224299220,21cyclictest0-21swapper/022:31:090
2512256992119,1cyclictest3056-21PK-Backend21:23:223
2512256992119,1cyclictest3056-21PK-Backend20:58:213
2512256992119,1cyclictest0-21swapper/323:58:223
2512256992119,1cyclictest0-21swapper/322:52:443
2512256992119,1cyclictest0-21swapper/322:29:323
2512256992119,1cyclictest0-21swapper/320:39:143
2512256992117,3cyclictest0-21swapper/319:40:213
251225699211,18cyclictest2584156-21df22:30:113
251225699210,20cyclictest3056-21PK-Backend23:38:013
251225699210,20cyclictest0-21swapper/323:17:313
251225699210,20cyclictest0-21swapper/323:01:373
251225699210,20cyclictest0-21swapper/319:54:133
251225699210,19cyclictest3056-21PK-Backend21:53:443
251225299214,7cyclictest34-21ksoftirqd/219:20:112
2512252992119,1cyclictest0-21swapper/223:39:032
2512252992119,1cyclictest0-21swapper/222:03:242
2512252992119,1cyclictest0-21swapper/221:11:532
2512252992119,1cyclictest0-21swapper/219:50:292
2512252992117,3cyclictest331rcuc/222:26:042
2512252992117,3cyclictest331rcuc/220:51:332
2512252992117,3cyclictest331rcuc/220:05:092
2512252992117,3cyclictest331rcuc/200:20:562
2512252992117,3cyclictest331rcuc/200:16:512
2512252992117,3cyclictest331rcuc/200:03:082
2512252992113,6cyclictest2586597-21awk22:35:202
251225299210,20cyclictest2544314-21needreboot20:35:172
251225299210,20cyclictest0-21swapper/221:27:292
251225299210,20cyclictest0-21swapper/221:09:512
251225299210,20cyclictest0-21swapper/219:29:132
251225299210,20cyclictest0-21swapper/200:30:412
2512249992119,1cyclictest3056-21PK-Backend23:00:551
2512249992119,1cyclictest3056-21PK-Backend22:55:271
2512249992119,1cyclictest3056-21PK-Backend22:41:401
2512249992119,1cyclictest3056-21PK-Backend22:35:061
2512249992119,1cyclictest3056-21PK-Backend19:40:051
2512249992119,1cyclictest3056-21PK-Backend19:35:531
2512249992119,1cyclictest3056-21PK-Backend19:35:531
2512249992119,1cyclictest0-21swapper/123:44:411
2512249992119,1cyclictest0-21swapper/121:56:121
2512249992119,1cyclictest0-21swapper/121:38:141
2512249992119,1cyclictest0-21swapper/120:50:281
2512249992118,2cyclictest3056-21PK-Backend19:56:551
251224999210,20cyclictest3056-21PK-Backend20:42:311
251224999210,19cyclictest3056-21PK-Backend22:54:191
251224999210,19cyclictest3056-21PK-Backend20:45:221
251224999210,19cyclictest3056-21PK-Backend20:39:401
251224999210,19cyclictest3056-21PK-Backend20:08:171
251224999210,19cyclictest3056-21PK-Backend00:38:291
251224999210,19cyclictest3056-21PK-Backend00:34:221
251224999210,19cyclictest3056-21PK-Backend00:05:021
251224999210,19cyclictest0-21swapper/123:37:241
2512242992119,1cyclictest3056-21PK-Backend20:10:430
2512242992119,1cyclictest0-21swapper/022:48:390
2512242992119,1cyclictest0-21swapper/022:22:290
2512242992119,1cyclictest0-21swapper/021:54:310
2512242992119,1cyclictest0-21swapper/021:44:040
2512242992119,1cyclictest0-21swapper/021:13:420
2512242992119,1cyclictest0-21swapper/020:39:490
2512242992119,1cyclictest0-21swapper/020:32:380
2512242992119,1cyclictest0-21swapper/019:19:460
2512242992119,1cyclictest0-21swapper/000:26:140
2512242992119,1cyclictest0-21swapper/000:05:040
2512242992119,1cyclictest0-21swapper/000:02:000
251224299210,20cyclictest3056-21PK-Backend20:00:480
251224299210,20cyclictest0-21swapper/023:01:170
251224299210,20cyclictest0-21swapper/019:51:430
251225699202,13cyclictest2605374-21apt23:30:113
251225699202,13cyclictest2605374-21apt23:30:113
2512256992018,1cyclictest3056-21PK-Backend23:47:463
2512256992018,1cyclictest3056-21PK-Backend21:36:543
2512256992018,1cyclictest0-21swapper/320:06:083
2512256992018,1cyclictest0-21swapper/320:00:553
2512256992018,1cyclictest0-21swapper/319:47:443
2512256992018,1cyclictest0-21swapper/319:13:363
2512256992018,1cyclictest0-21swapper/300:19:033
2512256992018,1cyclictest0-21swapper/300:10:383
2512256992018,1cyclictest0-21swapper/300:07:353
2512256992015,3cyclictest2623858-21python300:20:213
2512256992015,3cyclictest2602644-21ntp_states23:20:183
2512256992012,6cyclictest2613579-21ls23:50:213
251225699201,16cyclictest2598651-21irqstats23:10:153
251225699200,1cyclictest3056-21PK-Backend21:30:513
251225699200,19cyclictest3056-21PK-Backend21:10:113
251225699200,19cyclictest0-21swapper/322:43:193
251225699200,19cyclictest0-21swapper/322:02:013
251225699200,19cyclictest0-21swapper/319:58:193
251225699200,18cyclictest2549797-21/usr/sbin/munin20:50:203
2512252992017,2cyclictest331rcuc/222:58:482
2512252992016,3cyclictest331rcuc/223:31:442
2512252992016,3cyclictest331rcuc/223:31:432
2512252992016,3cyclictest331rcuc/220:55:172
2512252992015,4cyclictest34-21ksoftirqd/221:50:012
2512249992018,1cyclictest3056-21PK-Backend22:30:461
2512249992018,1cyclictest3056-21PK-Backend20:12:021
2512249992018,1cyclictest0-21swapper/121:23:471
251224999200,19cyclictest3056-21PK-Backend19:45:591
251224999200,19cyclictest0-21swapper/122:18:221
251224999200,19cyclictest0-21swapper/119:15:381
2512242992018,1cyclictest0-21swapper/023:53:220
2512242992018,1cyclictest0-21swapper/023:45:170
2512242992018,1cyclictest0-21swapper/023:06:440
2512242992018,1cyclictest0-21swapper/020:05:290
2512242992018,1cyclictest0-21swapper/000:19:240
2512242992018,1cyclictest0-21swapper/000:14:370
2512242992017,2cyclictest3056-21PK-Backend19:55:140
2512242992012,6cyclictest2558292-21awk21:15:200
251224299200,7cyclictest2564461-21idleruntime-cro21:34:590
251224299200,19cyclictest0-21swapper/023:33:010
251224299200,19cyclictest0-21swapper/023:33:010
251224299200,19cyclictest0-21swapper/022:54:400
251224299200,19cyclictest0-21swapper/021:35:390
251224299200,19cyclictest0-21swapper/000:30:180
139099200,3rtkit-daemon1389-21rtkit-daemon22:02:490
2512256991917,1cyclictest3056-21PK-Backend22:14:323
2512256991917,1cyclictest3056-21PK-Backend22:07:163
2512256991917,1cyclictest0-21swapper/322:45:353
2512256991917,1cyclictest0-21swapper/322:15:073
2512256991917,1cyclictest0-21swapper/300:00:523
2512256991914,3cyclictest2627202-21ntp_states00:30:183
2512256991914,3cyclictest2593088-21df_inode22:55:143
2512256991912,5cyclictest2603939-21expr23:25:143
251225699191,16cyclictest2523811-21munin-run19:40:003
251225699191,16cyclictest2523811-21munin-run19:39:593
251225699190,5cyclictest2624869-21grep00:25:133
251225699190,1cyclictest0-21swapper/320:22:393
251225699190,18cyclictest3056-21PK-Backend21:46:143
251225699190,18cyclictest0-21swapper/319:16:483
2512252991917,1cyclictest0-21swapper/222:22:172
2512252991917,1cyclictest0-21swapper/221:20:052
2512252991912,5cyclictest2563426-21awk21:30:142
251225299190,1cyclictest3056-21PK-Backend22:15:212
251225299190,1cyclictest0-21swapper/221:17:282
2512249991917,1cyclictest3056-21PK-Backend20:22:431
2512249991917,1cyclictest3056-21PK-Backend00:24:501
2512249991917,1cyclictest3056-21PK-Backend00:18:141
2512249991917,1cyclictest0-21swapper/121:03:331
2512249991917,1cyclictest0-21swapper/120:16:221
2512249991912,5cyclictest2582783-21ntpq22:25:181
2512249991911,6cyclictest2554588-21sed21:05:171
251224999190,1cyclictest3056-21PK-Backend23:34:201
251224999190,1cyclictest3056-21PK-Backend23:34:201
251224999190,1cyclictest3056-21PK-Backend00:27:331
251224999190,1cyclictest0-21swapper/121:19:081
251224999190,1cyclictest0-21swapper/120:55:141
251224999190,18cyclictest0-21swapper/121:10:021
2512242991917,1cyclictest0-21swapper/023:35:300
2512242991917,1cyclictest0-21swapper/023:14:420
2512242991917,1cyclictest0-21swapper/022:36:020
2512242991917,1cyclictest0-21swapper/022:15:340
2512242991917,1cyclictest0-21swapper/022:10:200
2512242991917,1cyclictest0-21swapper/022:09:460
2512242991917,1cyclictest0-21swapper/021:24:310
2512242991917,1cyclictest0-21swapper/019:39:190
2512242991917,1cyclictest0-21swapper/019:39:190
2512242991917,1cyclictest0-21swapper/019:13:240
251224299190,1cyclictest0-21swapper/020:44:220
2512256991816,1cyclictest3056-21PK-Backend21:41:073
2512256991811,5cyclictest2554249-21cut21:05:123
2512256991811,5cyclictest2541989-21cat20:30:123
2512256991810,6cyclictest2541267-21users20:25:223
2512249991816,1cyclictest3056-21PK-Backend19:13:011
2512249991816,1cyclictest0-21swapper/123:49:421
2512249991816,1cyclictest0-21swapper/121:40:381
2512249991816,1cyclictest0-21swapper/121:34:021
2512249991815,2cyclictest3056-21PK-Backend22:48:501
2512249991815,2cyclictest3056-21PK-Backend20:31:561
2512249991815,2cyclictest3056-21PK-Backend00:13:281
2512242991816,1cyclictest0-21swapper/023:29:560
2512242991816,1cyclictest0-21swapper/019:34:320
2512242991811,5cyclictest2562235-21cat21:25:190
2512242991810,6cyclictest1-21systemd22:45:010
2512256991715,1cyclictest3056-21PK-Backend21:01:173
2512256991715,1cyclictest0-21swapper/321:55:123
251225699171,3cyclictest2547690-21latency20:45:183
251225299170,5cyclictest0-21swapper/221:38:232
2512249991715,1cyclictest0-21swapper/122:11:051
2512249991714,2cyclictest3056-21PK-Backend00:00:491
2512249991710,5cyclictest2613006-21awk23:50:151
251224299174,11cyclictest2519800-21idleruntime-cro19:30:000
2512242991715,1cyclictest0-21swapper/020:55:240
2512242991710,5cyclictest2623512-21wc00:20:170
139099170,14rtkit-daemon0-21swapper/122:01:581
2512249991614,1cyclictest3056-21PK-Backend23:06:551
2512249991614,1cyclictest3056-21PK-Backend20:27:031
2512249991610,4cyclictest2518427-21cpuspeed_turbos19:25:131
251224299168,6cyclictest2547277-21tr20:45:130
2512242991614,1cyclictest0-21swapper/022:27:230
2512242991614,1cyclictest0-21swapper/020:27:450
251224999158,5cyclictest2560722-21apt-get21:25:021
251224999157,6cyclictest2570135-21cat21:50:131
2512249991510,4cyclictest2600816-21needreboot23:15:161
251224299150,5cyclictest0-21swapper/023:15:170
251224999137,5cyclictest2517865-21seq19:22:531
2512249991310,2cyclictest3056-21PK-Backend23:24:471
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional