You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-23 - 14:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Mon Mar 23, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
247814522500,19sleep20-21swapper/219:05:172
257543621010,2sleep00-21swapper/023:40:160
257543621010,2sleep00-21swapper/023:40:160
139099790,66rtkit-daemon0-21swapper/119:06:021
247767927845,22sleep30-21swapper/319:05:113
247854627045,17sleep00-21swapper/019:05:250
25496722500,5sleep23056-21PK-Backend22:25:202
25584712490,3sleep10-21swapper/122:50:221
25463612400,4sleep234-21ksoftirqd/222:15:222
25561312390,2sleep20-21swapper/222:45:132
25224082370,2sleep00-21swapper/021:10:150
2479266992912,14cyclictest2564247-21idleruntime-cro23:10:003
139099290,4rtkit-daemon0-21swapper/023:20:210
2479265992713,12cyclictest2565991-21unin-run23:15:012
2479266992618,5cyclictest2531372-21python321:35:193
247926699259,10cyclictest978-21snmpd00:05:033
247926699252,3cyclictest0-21swapper/319:50:003
25151772240,2sleep30-21swapper/320:50:133
25047512240,3sleep1251rcuc/120:20:121
247926699243,18cyclictest2486640-21sh19:30:003
247926599240,22cyclictest2525687-21diskstats21:20:142
25030392230,4sleep33056-21PK-Backend20:15:143
249208622318,3sleep02492110-21turbostat19:44:590
247926699231,19cyclictest2541434-21apt-get22:05:043
247926699231,18cyclictest2553227-21python322:35:203
247926699231,18cyclictest2553227-21python322:35:193
2479265992321,1cyclictest0-21swapper/219:33:082
2479261992321,1cyclictest0-21swapper/120:00:201
2479261992320,2cyclictest3056-21PK-Backend00:28:561
2479257992321,1cyclictest0-21swapper/020:14:170
247925799230,4cyclictest2156-21gnome-shell21:37:580
2479266992213,7cyclictest2520692-21/usr/sbin/munin21:05:193
247926699220,20cyclictest3056-21PK-Backend23:53:213
247926699220,20cyclictest3056-21PK-Backend23:37:063
247926699220,20cyclictest3056-21PK-Backend23:28:343
2479265992220,1cyclictest0-21swapper/223:23:102
2479265992220,1cyclictest0-21swapper/223:05:532
2479265992220,1cyclictest0-21swapper/200:08:122
2479265992220,1cyclictest0-21swapper/200:03:202
2479265992214,6cyclictest2562481-21cron23:05:002
247926599220,21cyclictest0-21swapper/200:16:572
2479261992214,6cyclictest2579432-21users23:50:211
2479261992213,6cyclictest2509334-21otd-news20:33:471
247926199220,21cyclictest3056-21PK-Backend21:18:351
2479257992220,1cyclictest0-21swapper/022:34:430
2479257992220,1cyclictest0-21swapper/022:24:220
2479257992220,1cyclictest0-21swapper/020:53:440
2479257992220,1cyclictest0-21swapper/020:46:410
2479257992220,1cyclictest0-21swapper/020:46:410
2479257992220,1cyclictest0-21swapper/020:22:070
2479257992220,1cyclictest0-21swapper/000:28:050
2479257992219,2cyclictest0-21swapper/023:32:360
2479257992215,5cyclictest2484893-21cron19:25:010
2479257992214,6cyclictest2529517-21ntp_states21:30:200
139099220,15rtkit-daemon3056-21PK-Backend20:55:251
139099220,14rtkit-daemon0-21swapper/120:44:301
247926699217,3cyclictest2592654-21latency_hist00:30:003
2479266992118,2cyclictest0-21swapper/300:34:243
2479266992113,5cyclictest2519241-21python321:00:223
247926699210,19cyclictest3056-21PK-Backend23:57:133
247926699210,19cyclictest3056-21PK-Backend23:31:243
247926699210,19cyclictest3056-21PK-Backend22:51:503
247926699210,19cyclictest3056-21PK-Backend22:21:153
247926699210,19cyclictest3056-21PK-Backend22:16:353
2479265992119,1cyclictest3056-21PK-Backend20:25:472
2479265992119,1cyclictest0-21swapper/223:47:372
2479265992119,1cyclictest0-21swapper/223:41:422
2479265992119,1cyclictest0-21swapper/223:41:422
2479265992119,1cyclictest0-21swapper/223:33:452
2479265992119,1cyclictest0-21swapper/222:44:582
2479265992119,1cyclictest0-21swapper/222:34:372
2479265992119,1cyclictest0-21swapper/222:02:332
2479265992119,1cyclictest0-21swapper/221:44:562
2479265992119,1cyclictest0-21swapper/221:18:252
2479265992119,1cyclictest0-21swapper/221:09:192
2479265992119,1cyclictest0-21swapper/220:59:262
2479265992119,1cyclictest0-21swapper/220:46:082
2479265992119,1cyclictest0-21swapper/220:46:072
2479265992119,1cyclictest0-21swapper/220:15:182
2479265992119,1cyclictest0-21swapper/220:06:402
2479265992119,1cyclictest0-21swapper/219:44:572
2479265992119,1cyclictest0-21swapper/219:12:252
2479265992119,1cyclictest0-21swapper/200:30:372
2479265992114,5cyclictest2592631-21cut00:30:002
247926599210,20cyclictest0-21swapper/223:52:302
247926599210,20cyclictest0-21swapper/222:09:012
247926599210,20cyclictest0-21swapper/221:56:042
247926599210,20cyclictest0-21swapper/221:35:492
247926599210,20cyclictest0-21swapper/220:04:042
2479261992119,1cyclictest3056-21PK-Backend21:48:151
2479261992119,1cyclictest3056-21PK-Backend19:17:081
2479261992119,1cyclictest3056-21PK-Backend00:39:171
2479261992119,1cyclictest0-21swapper/123:18:121
2479261992119,1cyclictest0-21swapper/122:46:221
2479261992119,1cyclictest0-21swapper/122:25:071
2479261992119,1cyclictest0-21swapper/122:20:261
2479261992119,1cyclictest0-21swapper/122:17:361
2479261992119,1cyclictest0-21swapper/122:12:561
2479261992119,1cyclictest0-21swapper/120:45:561
2479261992119,1cyclictest0-21swapper/120:45:561
2479261992119,1cyclictest0-21swapper/120:06:571
2479261992119,1cyclictest0-21swapper/119:57:571
2479261992119,1cyclictest0-21swapper/119:53:581
2479261992119,1cyclictest0-21swapper/119:24:041
2479261992118,2cyclictest3056-21PK-Backend00:00:311
2479261992115,5cyclictest2574656-21systemd-run23:40:001
2479261992112,7cyclictest436-20systemd-journal20:15:011
247926199210,20cyclictest3056-21PK-Backend22:33:241
247926199210,20cyclictest3056-21PK-Backend21:32:071
247926199210,20cyclictest0-21swapper/123:45:031
247926199210,20cyclictest0-21swapper/120:26:371
247926199210,20cyclictest0-21swapper/100:17:341
247926199210,20cyclictest0-21swapper/100:11:541
2479257992119,1cyclictest0-21swapper/023:36:280
2479257992119,1cyclictest0-21swapper/022:25:230
2479257992119,1cyclictest0-21swapper/020:27:210
2479257992119,1cyclictest0-21swapper/019:36:450
2479257992119,1cyclictest0-21swapper/000:07:100
2479257992116,4cyclictest2519051-21head21:00:200
247925799210,20cyclictest0-21swapper/023:06:470
247925799210,20cyclictest0-21swapper/022:40:240
247925799210,20cyclictest0-21swapper/020:04:030
247925799210,20cyclictest0-21swapper/019:45:590
247925799210,20cyclictest0-21swapper/019:30:090
139099210,15rtkit-daemon3056-21PK-Backend23:14:231
139099210,15rtkit-daemon3056-21PK-Backend21:08:481
139099210,14rtkit-daemon3056-21PK-Backend23:06:061
139099210,14rtkit-daemon3056-21PK-Backend22:07:251
139099210,14rtkit-daemon0-21swapper/022:07:100
139099210,13rtkit-daemon0-21swapper/119:31:311
25506242200,2sleep33056-21PK-Backend22:30:103
2479266992018,1cyclictest3056-21PK-Backend20:40:513
2479266992018,1cyclictest0-21swapper/323:04:013
2479266992018,1cyclictest0-21swapper/322:01:483
2479266992018,1cyclictest0-21swapper/319:14:113
2479266992017,2cyclictest3056-21PK-Backend20:49:283
2479266992017,2cyclictest3056-21PK-Backend20:49:283
2479266992015,3cyclictest978-21snmpd21:25:403
2479266992013,5cyclictest2536698-21interrupts21:50:163
2479266992011,7cyclictest2523669-21ls21:15:113
247926699200,19cyclictest3056-21PK-Backend20:07:313
247926699200,19cyclictest3056-21PK-Backend19:33:033
247926699200,19cyclictest0-21swapper/321:32:283
247926699200,19cyclictest0-21swapper/321:23:223
247926699200,19cyclictest0-21swapper/300:38:163
2479265992018,1cyclictest0-21swapper/223:16:272
2479265992018,1cyclictest0-21swapper/222:53:292
2479265992018,1cyclictest0-21swapper/222:35:382
2479265992018,1cyclictest0-21swapper/222:35:382
2479265992018,1cyclictest0-21swapper/222:23:142
247926599200,19cyclictest0-21swapper/221:53:132
247926599200,19cyclictest0-21swapper/221:11:102
247926599200,19cyclictest0-21swapper/221:01:152
247926599200,19cyclictest0-21swapper/220:35:262
247926599200,19cyclictest0-21swapper/220:31:002
247926599200,19cyclictest0-21swapper/220:13:292
247926599200,19cyclictest0-21swapper/219:38:072
247926599200,19cyclictest0-21swapper/219:24:082
2479261992018,1cyclictest3056-21PK-Backend21:59:101
2479261992018,1cyclictest0-21swapper/123:29:351
2479261992018,1cyclictest0-21swapper/122:59:201
2479261992018,1cyclictest0-21swapper/120:17:581
2479261992018,1cyclictest0-21swapper/119:46:141
2479261992017,2cyclictest3056-21PK-Backend22:36:151
2479261992017,2cyclictest3056-21PK-Backend22:36:151
2479261992017,2cyclictest3056-21PK-Backend21:42:001
247926199200,1cyclictest3056-21PK-Backend22:01:001
247926199200,1cyclictest0-21swapper/100:06:121
247926199200,19cyclictest3056-21PK-Backend23:02:101
247926199200,19cyclictest0-21swapper/123:30:361
247926199200,19cyclictest0-21swapper/120:53:001
247926199200,19cyclictest0-21swapper/119:40:271
247926199200,19cyclictest0-21swapper/119:28:501
2479257992018,1cyclictest0-21swapper/023:50:540
2479257992018,1cyclictest0-21swapper/023:25:070
2479257992018,1cyclictest0-21swapper/021:26:160
2479257992018,1cyclictest0-21swapper/020:44:530
2479257992014,4cyclictest436-20systemd-journal21:10:000
247925799200,1cyclictest0-21swapper/023:10:390
247925799200,1cyclictest0-21swapper/023:00:040
247925799200,1cyclictest0-21swapper/020:35:120
247925799200,1cyclictest0-21swapper/000:36:370
247925799200,19cyclictest0-21swapper/022:39:240
247925799200,19cyclictest0-21swapper/022:39:230
247925799200,19cyclictest0-21swapper/021:50:560
247925799200,19cyclictest0-21swapper/021:42:380
247925799200,19cyclictest0-21swapper/020:16:550
247925799200,19cyclictest0-21swapper/000:12:510
139099200,14rtkit-daemon0-21swapper/120:38:451
139099200,14rtkit-daemon0-21swapper/019:55:560
139099200,14rtkit-daemon0-21swapper/019:29:520
139099200,13rtkit-daemon0-21swapper/119:36:031
139099200,13rtkit-daemon0-21swapper/020:55:080
139099200,13rtkit-daemon0-21swapper/019:17:130
247926699194,13cyclictest2582249-21idleruntime00:00:163
2479266991912,5cyclictest2555933-21cat22:45:123
2479266991912,4cyclictest2481512-21python319:15:023
247926699190,1cyclictest3056-21PK-Backend23:40:583
247926699190,1cyclictest3056-21PK-Backend23:40:583
247926699190,1cyclictest3056-21PK-Backend20:39:013
247926699190,1cyclictest3056-21PK-Backend20:12:313
247926699190,1cyclictest3056-21PK-Backend19:23:313
247926699190,1cyclictest3056-21PK-Backend00:24:513
247926699190,18cyclictest3056-21PK-Backend19:51:373
247926699190,18cyclictest3056-21PK-Backend00:11:403
247926699190,18cyclictest0-21swapper/323:13:333
2479265991918,1cyclictest0-21swapper/200:24:412
2479265991917,1cyclictest0-21swapper/223:58:252
2479265991917,1cyclictest0-21swapper/223:35:482
2479265991917,1cyclictest0-21swapper/223:27:022
2479265991917,1cyclictest0-21swapper/222:14:432
2479265991917,1cyclictest0-21swapper/221:32:112
2479265991917,1cyclictest0-21swapper/220:20:322
2479265991917,1cyclictest0-21swapper/200:37:192
2479265991917,1cyclictest0-21swapper/200:10:152
247926599190,1cyclictest0-21swapper/222:58:092
247926599190,1cyclictest0-21swapper/221:29:202
247926599190,1cyclictest0-21swapper/220:50:342
247926599190,1cyclictest0-21swapper/220:42:292
247926599190,1cyclictest0-21swapper/219:18:342
2479261991917,1cyclictest0-21swapper/123:56:391
247926199190,1cyclictest3056-21PK-Backend19:14:451
247926199190,1cyclictest3056-21PK-Backend00:33:361
247926199190,1cyclictest0-21swapper/123:41:121
247926199190,1cyclictest0-21swapper/123:41:111
247926199190,1cyclictest0-21swapper/123:21:031
247926199190,18cyclictest3056-21PK-Backend21:02:391
2479257991917,1cyclictest0-21swapper/023:57:370
2479257991917,1cyclictest0-21swapper/022:57:140
2479257991917,1cyclictest0-21swapper/022:54:230
2479257991917,1cyclictest0-21swapper/021:55:360
2479257991917,1cyclictest0-21swapper/021:45:290
2479257991917,1cyclictest0-21swapper/021:20:480
2479257991917,1cyclictest0-21swapper/020:06:390
2479257991917,1cyclictest0-21swapper/019:53:420
2479257991917,1cyclictest0-21swapper/000:01:290
2479257991912,5cyclictest883-21dbus-daemon22:48:030
2479257991912,5cyclictest2577295-21needreboot23:45:180
2479257991911,6cyclictest2566926-21ntp_kernel_err23:15:150
247925799190,1cyclictest0-21swapper/022:15:030
247925799190,1cyclictest0-21swapper/020:33:230
2479266991816,1cyclictest3056-21PK-Backend23:23:543
2479266991816,1cyclictest3056-21PK-Backend22:27:443
2479266991816,1cyclictest3056-21PK-Backend20:29:343
2479266991816,1cyclictest3056-21PK-Backend19:44:393
2479266991816,1cyclictest0-21swapper/321:13:283
2479266991816,1cyclictest0-21swapper/320:58:213
247926699181,3cyclictest2554963-21python322:40:213
247926699181,3cyclictest2544557-21python322:10:203
2479266991812,4cyclictest2490360-21sh19:39:593
2479266991811,5cyclictest2538415-21grep21:55:123
247926699181,12cyclictest2534240-21apt21:45:123
247926699180,1cyclictest3056-21PK-Backend22:56:303
2479265991816,1cyclictest0-21swapper/219:47:202
2479265991812,4cyclictest34-21ksoftirqd/221:45:142
2479261991816,1cyclictest3056-21PK-Backend21:25:511
2479261991816,1cyclictest0-21swapper/121:35:451
2479261991816,1cyclictest0-21swapper/100:21:261
247925799189,7cyclictest2013-21gdbus22:03:230
2479257991816,1cyclictest3056-21PK-Backend21:15:210
2479257991816,1cyclictest0-21swapper/019:14:140
247926699171,15cyclictest3056-21PK-Backend23:50:003
2479266991710,5cyclictest2566437-21ls23:15:103
2479265991711,5cyclictest2486640-21cron19:30:012
2479261991714,2cyclictest3056-21PK-Backend21:24:031
2479257991715,1cyclictest0-21swapper/000:31:570
2479257991715,1cyclictest0-21swapper/000:20:360
25228872160,5sleep13056-21PK-Backend21:10:201
247926699169,5cyclictest436-20systemd-journal21:44:563
2479261991614,1cyclictest3056-21PK-Backend21:51:071
2479257991614,1cyclictest0-21swapper/022:13:130
2479257991614,1cyclictest0-21swapper/000:19:340
2479266991513,1cyclictest3056-21PK-Backend20:21:573
2479266991513,1cyclictest3056-21PK-Backend00:18:223
2479261991511,2cyclictest26-21ksoftirqd/122:42:351
247926699140,2cyclictest3056-21PK-Backend20:05:003
247926599142,11cyclictest2495554-21cron19:54:592
2479265991410,3cyclictest2496481-21tail19:55:182
247926699139,3cyclictest3056-21PK-Backend19:57:403
2479266991210,1cyclictest3056-21PK-Backend20:31:243
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional