You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 19:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Jun 30, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139099770,67rtkit-daemon0-21swapper/307:09:303
1968212690,3sleep10-21swapper/112:32:461
8155726548,10sleep20-21swapper/207:10:012
8139026343,14sleep00-21swapper/007:07:530
8087826341,14sleep10-21swapper/107:05:251
1664612520,4sleep0166451-21apt-config11:10:010
1233872470,3sleep00-21swapper/009:05:180
81698993213,17cyclictest91135-21turbostat.cron07:39:561
81698993110,6cyclictest100725-21ls08:00:161
8169899298,5cyclictest436-20systemd-journal10:24:551
8169899289,5cyclictest188502-21date12:10:011
8169899289,5cyclictest172695-21cat11:25:131
8169899289,5cyclictest159845-21expr10:50:121
8169899289,4cyclictest128769-21tail09:20:171
8169899288,5cyclictest165519-21awk11:05:191
81698992810,6cyclictest99405-21cat07:55:211
8169699282,8cyclictest92848-21file07:39:550
81698992712,3cyclictest121260-21cpuspeed09:00:121
81696992719,1cyclictest0-21swapper/007:14:310
1851992270,11sleep234-21ksoftirqd/212:00:112
8169899269,3cyclictest157023-21ls10:40:211
8169899269,3cyclictest157023-21ls10:40:211
8169899268,6cyclictest182077-21idleruntime11:50:161
8169899268,5cyclictest197535-21ls12:35:131
8169899268,5cyclictest153709-21vmstat10:30:231
8169899267,5cyclictest177413-21threads11:35:201
81698992612,12cyclictest115477-21unin-run08:45:001
81698992610,3cyclictest105729-21idleruntime08:15:171
8170699250,23cyclictest978-21snmpd12:14:263
8169899259,3cyclictest132442-21swap09:30:221
8169899257,5cyclictest82053-21cut07:10:121
81698992511,3cyclictest94645-21latency_hist07:45:021
81698992510,3cyclictest179916-21/usr/sbin/munin11:45:131
81698992510,3cyclictest102299-21if_enp2s008:05:171
81698992510,2cyclictest168225-21latency_hist11:15:011
81698992510,2cyclictest119263-21apt-key08:55:001
8169899242,6cyclictest193806-21apt12:25:121
81698992411,5cyclictest26-21ksoftirqd/112:15:101
139099240,3rtkit-daemon1389-21rtkit-daemon11:38:202
81706992310,11cyclictest179666-21sh11:45:003
8169899239,12cyclictest131302-21dump-pmu-power09:30:001
81696992318,3cyclictest436-20systemd-journal07:34:550
81696992315,6cyclictest113623-21sshd08:39:000
139099230,4rtkit-daemon1389-21rtkit-daemon07:20:252
139099230,3rtkit-daemon1389-21rtkit-daemon11:58:583
81706992220,1cyclictest3056-21PK-Backend11:45:403
81706992220,1cyclictest3056-21PK-Backend11:28:023
81706992220,1cyclictest3056-21PK-Backend08:43:023
81706992219,2cyclictest3056-21PK-Backend07:44:223
8170699220,20cyclictest3056-21PK-Backend12:37:243
8170699220,20cyclictest3056-21PK-Backend09:22:173
8170699220,20cyclictest3056-21PK-Backend08:09:023
81701992220,1cyclictest0-21swapper/212:32:142
81701992220,1cyclictest0-21swapper/212:26:122
81701992220,1cyclictest0-21swapper/211:32:322
81701992220,1cyclictest0-21swapper/211:28:472
81701992220,1cyclictest0-21swapper/209:42:132
81701992219,2cyclictest133039-21sh09:35:002
8170199220,20cyclictest0-21swapper/210:46:502
8169899220,21cyclictest0-21swapper/111:01:181
81696992220,1cyclictest0-21swapper/009:43:510
81696992220,1cyclictest0-21swapper/008:43:210
81696992220,1cyclictest0-21swapper/008:08:180
81696992220,1cyclictest0-21swapper/008:03:450
81696992219,2cyclictest0-21swapper/010:13:180
8169699220,21cyclictest0-21swapper/011:22:180
8169699220,21cyclictest0-21swapper/011:15:430
8169699220,20cyclictest151258-21df10:25:130
139099220,4rtkit-daemon1389-21rtkit-daemon08:28:372
139099220,19rtkit-daemon2637-21gmain08:07:372
81706992119,1cyclictest3056-21PK-Backend10:31:043
81706992119,1cyclictest3056-21PK-Backend09:59:413
81706992119,1cyclictest3056-21PK-Backend09:05:543
81706992119,1cyclictest3056-21PK-Backend08:45:473
81706992119,1cyclictest3056-21PK-Backend07:57:143
81706992119,1cyclictest3056-21PK-Backend07:12:243
81706992119,1cyclictest0-21swapper/312:17:173
81706992118,2cyclictest3056-21PK-Backend09:51:233
81706992118,2cyclictest3056-21PK-Backend08:03:073
8170699210,20cyclictest3056-21PK-Backend07:26:453
8170699210,20cyclictest0-21swapper/311:37:023
8170699210,19cyclictest3056-21PK-Backend10:45:373
8170699210,19cyclictest3056-21PK-Backend10:41:333
8170699210,19cyclictest3056-21PK-Backend10:41:323
8170699210,19cyclictest3056-21PK-Backend09:00:403
8170699210,19cyclictest3056-21PK-Backend07:48:073
81701992119,1cyclictest3056-21PK-Backend11:12:112
81701992119,1cyclictest0-21swapper/212:11:532
81701992119,1cyclictest0-21swapper/210:42:102
81701992119,1cyclictest0-21swapper/210:42:092
81701992119,1cyclictest0-21swapper/209:07:052
81701992119,1cyclictest0-21swapper/208:01:162
81701992119,1cyclictest0-21swapper/207:51:212
81701992119,1cyclictest0-21swapper/207:30:402
8170199210,2cyclictest1445012sleep210:05:192
8170199210,20cyclictest0-21swapper/212:38:162
8170199210,20cyclictest0-21swapper/209:25:382
8170199210,20cyclictest0-21swapper/207:42:492
8170199210,1cyclictest0-21swapper/210:27:292
8169899211,7cyclictest126348-21/usr/sbin/munin09:15:241
8169899211,6cyclictest161306-21apt10:55:121
8169699217,12cyclictest99870-21unin-run08:00:010
81696992119,1cyclictest3056-21PK-Backend11:55:250
81696992119,1cyclictest0-21swapper/012:18:560
81696992119,1cyclictest0-21swapper/010:16:070
81696992119,1cyclictest0-21swapper/010:03:510
81696992119,1cyclictest0-21swapper/009:54:170
81696992119,1cyclictest0-21swapper/009:31:050
81696992119,1cyclictest0-21swapper/008:54:360
81696992119,1cyclictest0-21swapper/008:48:480
81696992119,1cyclictest0-21swapper/008:21:580
81696992119,1cyclictest0-21swapper/008:17:250
81696992119,1cyclictest0-21swapper/008:12:510
81696992119,1cyclictest0-21swapper/007:29:050
81696992119,1cyclictest0-21swapper/007:21:080
81696992119,1cyclictest0-21swapper/007:19:460
81696992115,4cyclictest137444-21grep09:45:180
81696992115,4cyclictest137444-21grep09:45:170
8169699210,20cyclictest0-21swapper/011:48:140
8169699210,20cyclictest0-21swapper/010:45:410
8169699210,20cyclictest0-21swapper/009:24:090
8169699210,20cyclictest0-21swapper/009:12:000
139099210,18rtkit-daemon81688-21cyclictest10:17:082
1090322210,4sleep00-21swapper/008:25:120
81706992018,1cyclictest3056-21PK-Backend11:16:333
81706992018,1cyclictest3056-21PK-Backend10:39:303
81706992018,1cyclictest3056-21PK-Backend08:38:573
81706992018,1cyclictest3056-21PK-Backend08:30:113
81706992018,1cyclictest3056-21PK-Backend08:29:253
81706992018,1cyclictest3056-21PK-Backend08:10:383
81706992018,1cyclictest0-21swapper/309:39:473
81706992018,1cyclictest0-21swapper/307:53:283
8170699201,1cyclictest3056-21PK-Backend11:30:403
8170699200,1cyclictest3056-21PK-Backend07:32:463
8170699200,1cyclictest3056-21PK-Backend07:20:093
8170699200,1cyclictest3056-21PK-Backend07:18:463
8170699200,19cyclictest3056-21PK-Backend11:22:013
8170699200,19cyclictest3056-21PK-Backend11:11:063
8170699200,19cyclictest3056-21PK-Backend10:05:443
8170699200,19cyclictest3056-21PK-Backend09:11:293
8170699200,18cyclictest3056-21PK-Backend09:27:513
8170699200,18cyclictest3056-21PK-Backend08:21:063
81701992018,1cyclictest0-21swapper/211:44:222
81701992018,1cyclictest0-21swapper/210:36:012
81701992018,1cyclictest0-21swapper/209:36:252
81701992018,1cyclictest0-21swapper/209:23:082
81701992018,1cyclictest0-21swapper/208:58:342
81701992018,1cyclictest0-21swapper/208:51:252
81701992018,1cyclictest0-21swapper/208:20:142
81701992018,1cyclictest0-21swapper/208:18:392
81701992018,1cyclictest0-21swapper/207:26:342
81701992012,6cyclictest180072-21cut11:45:122
8170199200,1cyclictest0-21swapper/212:09:372
8170199200,1cyclictest0-21swapper/211:20:292
8170199200,1cyclictest0-21swapper/210:52:032
8170199200,19cyclictest3056-21PK-Backend11:50:232
8170199200,19cyclictest3056-21PK-Backend10:04:442
8170199200,19cyclictest0-21swapper/210:56:222
8170199200,19cyclictest0-21swapper/209:11:312
8170199200,19cyclictest0-21swapper/207:48:582
8170199200,19cyclictest0-21swapper/207:37:302
8169899204,15cyclictest145184-21sh10:10:001
81698992018,1cyclictest0-21swapper/108:13:351
81698992018,1cyclictest0-21swapper/107:52:251
81698992015,4cyclictest0-21swapper/108:30:241
81698992013,1cyclictest0-21swapper/111:23:091
8169899200,1cyclictest3056-21PK-Backend12:03:501
8169899200,1cyclictest0-21swapper/110:14:401
8169899200,19cyclictest3056-21PK-Backend12:12:291
8169899200,19cyclictest0-21swapper/111:34:571
8169899200,19cyclictest0-21swapper/110:19:471
8169899200,19cyclictest0-21swapper/110:01:301
8169899200,19cyclictest0-21swapper/109:51:281
8169899200,19cyclictest0-21swapper/109:48:101
8169899200,19cyclictest0-21swapper/109:48:101
8169899200,19cyclictest0-21swapper/108:28:001
81696992018,1cyclictest3056-21PK-Backend11:40:440
81696992018,1cyclictest3056-21PK-Backend11:36:240
81696992018,1cyclictest0-21swapper/012:12:550
81696992018,1cyclictest0-21swapper/011:28:550
81696992018,1cyclictest0-21swapper/009:35:330
81696992018,1cyclictest0-21swapper/009:15:520
81696992012,6cyclictest95616-21ntpq07:45:190
8169699201,17cyclictest141703-21awk10:00:000
8169699200,1cyclictest0-21swapper/011:00:290
8169699200,1cyclictest0-21swapper/010:35:280
8169699200,19cyclictest0-21swapper/012:37:210
8169699200,19cyclictest0-21swapper/012:32:280
8169699200,19cyclictest0-21swapper/012:00:520
8169699200,19cyclictest0-21swapper/011:14:340
8169699200,19cyclictest0-21swapper/010:59:540
8169699200,18cyclictest3056-21PK-Backend07:41:490
139099200,3rtkit-daemon1389-21rtkit-daemon08:46:232
8170699195,2cyclictest166458-21seq11:10:023
81706991917,1cyclictest3056-21PK-Backend11:02:283
81706991917,1cyclictest3056-21PK-Backend10:58:423
81706991917,1cyclictest3056-21PK-Backend10:15:093
81706991917,1cyclictest3056-21PK-Backend08:56:483
81706991917,1cyclictest3056-21PK-Backend08:17:213
81706991917,1cyclictest0-21swapper/309:45:363
81706991917,1cyclictest0-21swapper/309:45:363
81706991911,6cyclictest118917-21apt-get08:55:003
8170699190,1cyclictest3056-21PK-Backend12:27:393
8170699190,1cyclictest3056-21PK-Backend12:22:443
8170699190,1cyclictest3056-21PK-Backend10:54:573
8170699190,1cyclictest3056-21PK-Backend10:12:193
8170699190,1cyclictest3056-21PK-Backend09:33:253
8170699190,1cyclictest0-21swapper/310:04:343
81701991917,1cyclictest0-21swapper/212:21:192
81701991917,1cyclictest0-21swapper/211:19:552
81701991917,1cyclictest0-21swapper/209:55:252
81701991916,2cyclictest1095-21runrttasks12:16:252
8170199190,1cyclictest0-21swapper/209:48:022
8170199190,1cyclictest0-21swapper/209:48:012
8170199190,1cyclictest0-21swapper/209:03:482
8170199190,1cyclictest0-21swapper/207:56:432
8170199190,18cyclictest0-21swapper/208:37:452
81698991917,1cyclictest0-21swapper/109:44:181
81698991917,1cyclictest0-21swapper/107:29:471
81698991914,3cyclictest118860-21runrttasks08:54:301
81698991911,6cyclictest84835-21ntpq07:15:191
81698991911,6cyclictest134772-21cron09:40:001
8169899190,1cyclictest0-21swapper/112:20:331
8169899190,1cyclictest0-21swapper/109:11:201
8169899190,18cyclictest0-21swapper/108:37:481
81696991917,1cyclictest0-21swapper/010:20:060
81696991917,1cyclictest0-21swapper/009:26:050
81696991912,5cyclictest97405-21awk07:50:200
81696991912,5cyclictest436-20systemd-journal09:04:590
81696991912,5cyclictest182414-21ntpq11:50:190
81696991912,5cyclictest119280-21mktemp08:55:000
81696991910,7cyclictest153787-21gdbus10:31:130
8169699190,1cyclictest0-21swapper/012:23:160
8169699190,1cyclictest0-21swapper/008:33:470
139099190,17rtkit-daemon882-21avahi-daemon08:42:532
8170699183,14cyclictest1095-21runrttasks12:02:593
81706991816,1cyclictest3056-21PK-Backend07:39:493
81701991816,1cyclictest0-21swapper/211:56:592
81701991816,1cyclictest0-21swapper/211:06:432
81701991816,1cyclictest0-21swapper/209:17:192
81701991816,1cyclictest0-21swapper/208:31:442
81701991816,1cyclictest0-21swapper/207:16:402
81701991816,1cyclictest0-21swapper/207:10:532
81701991814,3cyclictest978-21snmpd10:34:552
81698991816,1cyclictest0-21swapper/111:16:321
81698991816,1cyclictest0-21swapper/110:37:451
81698991816,1cyclictest0-21swapper/107:47:531
81698991813,1cyclictest0-21swapper/108:23:271
81698991811,6cyclictest157969-21expr10:45:131
8169899180,1cyclictest0-21swapper/109:58:051
8169899180,16cyclictest87672-21find07:25:011
81696991816,1cyclictest0-21swapper/012:07:270
81696991816,1cyclictest0-21swapper/011:32:400
81696991816,1cyclictest0-21swapper/010:05:340
81706991715,1cyclictest3056-21PK-Backend11:53:443
8170699171,12cyclictest186871-21apt12:05:103
81701991715,1cyclictest0-21swapper/209:50:462
81698991715,1cyclictest0-21swapper/109:06:211
81698991711,5cyclictest183491-21ls11:55:121
81696991712,4cyclictest13-21ksoftirqd/010:55:000
81706991614,1cyclictest3056-21PK-Backend12:33:063
81706991614,1cyclictest3056-21PK-Backend09:17:513
81706991613,2cyclictest3056-21PK-Backend09:43:403
8170199162,12cyclictest150619-21cron10:25:012
81701991614,1cyclictest0-21swapper/208:14:062
81698991612,2cyclictest115451-21latency_hist08:45:001
8169699167,6cyclictest157475-21sshd10:44:010
8169699167,6cyclictest157475-21sshd10:44:000
1785552160,4sleep13056-21PK-Backend11:40:131
81706991513,1cyclictest3056-21PK-Backend10:20:163
81706991510,4cyclictest3056-21PK-Backend10:26:133
8170199159,4cyclictest913-21gdbus10:10:012
81701991513,1cyclictest0-21swapper/211:01:502
81698991510,4cyclictest1095-21runrttasks10:28:401
81698991510,3cyclictest90605-21timerwakeupswit07:30:211
8169699152,10cyclictest193936-21cpuspeed_turbos12:25:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional