You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-09 - 22:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Jun 09, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
81973321210,2sleep13056-21PK-Backend10:00:111
86031821130,3sleep1251rcuc/111:50:221
8594102830,1sleep3859412-21grep11:50:123
75741528046,23sleep10-21swapper/107:05:121
139099780,68rtkit-daemon0-21swapper/307:06:443
75863727150,14sleep00-21swapper/007:06:470
75857726141,14sleep20-21swapper/207:06:022
7720032460,3chrt0-21swapper/007:45:130
8574632420,4sleep13056-21PK-Backend11:45:011
8116102420,3sleep13056-21PK-Backend09:35:171
7812022410,4sleep13056-21PK-Backend08:10:121
139099310,3rtkit-daemon0-21swapper/212:15:212
139099290,20rtkit-daemon817322-21wc09:50:212
759041992713,12cyclictest834228-21unin-run10:40:003
7665462260,4chrt0-21swapper/007:30:010
139099260,17rtkit-daemon780333-21munin-run08:10:002
8351722240,1sleep30-21swapper/310:40:193
759041992417,6cyclictest832311-21mktemp10:35:013
75904199230,1cyclictest0-21swapper/308:46:453
759034992321,1cyclictest0-21swapper/208:14:502
759034992315,6cyclictest760698-21/usr/sbin/munin07:15:002
759026992315,6cyclictest784038-21apt-get08:20:000
139099230,2rtkit-daemon0-21swapper/209:09:012
759041992215,6cyclictest1-21systemd11:15:003
759041992214,6cyclictest850433-21df11:25:123
759034992220,1cyclictest0-21swapper/212:07:022
759034992220,1cyclictest0-21swapper/209:30:272
759034992220,1cyclictest0-21swapper/209:24:132
759034992220,1cyclictest0-21swapper/207:55:582
759034992220,1cyclictest0-21swapper/207:35:022
759034992215,4cyclictest803611-21dump-pmu-power09:15:002
759034992214,6cyclictest28650irq/123-ahci[0007:15:202
759034992213,7cyclictest828180-21unin-run10:25:002
75903499220,20cyclictest866936-21ntp_states12:10:202
75903499220,20cyclictest813682-21python309:40:212
759029992220,1cyclictest3056-21PK-Backend12:00:121
759029992220,1cyclictest3056-21PK-Backend11:30:311
759029992220,1cyclictest0-21swapper/110:41:101
759029992213,7cyclictest436-20systemd-journal07:30:001
75902999220,20cyclictest3056-21PK-Backend10:10:201
75902999220,20cyclictest3056-21PK-Backend09:47:561
75902999220,20cyclictest3056-21PK-Backend09:10:381
139099220,14rtkit-daemon0-21swapper/207:47:572
75904199212,4cyclictest870783-21python312:20:223
75904199210,6cyclictest3056-21PK-Backend08:55:003
75904199210,20cyclictest3056-21PK-Backend07:39:183
75904199210,19cyclictest3056-21PK-Backend08:16:233
759034992119,1cyclictest0-21swapper/210:39:232
759034992119,1cyclictest0-21swapper/210:32:472
759034992119,1cyclictest0-21swapper/210:01:092
759034992119,1cyclictest0-21swapper/209:58:052
759034992119,1cyclictest0-21swapper/209:39:192
759034992119,1cyclictest0-21swapper/208:35:312
759034992119,1cyclictest0-21swapper/208:19:162
759034992117,2cyclictest763585-21smartctl07:20:152
759034992116,3cyclictest874876-21idleruntime-cro12:35:002
759034992110,8cyclictest883-21dbus-daemon07:44:592
75903499210,20cyclictest0-21swapper/212:01:282
75903499210,20cyclictest0-21swapper/211:19:102
75903499210,20cyclictest0-21swapper/211:11:332
75903499210,20cyclictest0-21swapper/211:09:162
75903499210,20cyclictest0-21swapper/210:16:032
75903499210,20cyclictest0-21swapper/210:08:052
75903499210,20cyclictest0-21swapper/208:53:582
75903499210,20cyclictest0-21swapper/208:21:062
75903499210,20cyclictest0-21swapper/207:26:312
75902999213,12cyclictest832178-40unattended-upgr10:35:041
759029992119,1cyclictest3056-21PK-Backend12:32:171
759029992119,1cyclictest3056-21PK-Backend12:23:511
759029992119,1cyclictest3056-21PK-Backend11:24:091
759029992119,1cyclictest3056-21PK-Backend09:15:521
759029992119,1cyclictest3056-21PK-Backend07:51:301
759029992119,1cyclictest3056-21PK-Backend07:51:301
759029992119,1cyclictest3056-21PK-Backend07:18:171
759029992119,1cyclictest3056-21PK-Backend07:14:201
759029992119,1cyclictest0-21swapper/110:18:051
759029992119,1cyclictest0-21swapper/108:43:071
759029992119,1cyclictest0-21swapper/108:16:441
759029992118,2cyclictest3056-21PK-Backend09:51:491
759029992118,2cyclictest3056-21PK-Backend09:20:181
75902999210,20cyclictest3056-21PK-Backend11:08:071
75902999210,20cyclictest0-21swapper/109:00:111
75902999210,20cyclictest0-21swapper/108:05:421
75902999210,19cyclictest3056-21PK-Backend12:39:251
75902999210,19cyclictest3056-21PK-Backend12:09:511
75902999210,19cyclictest3056-21PK-Backend07:47:381
759026992119,1cyclictest0-21swapper/011:57:350
759026992119,1cyclictest0-21swapper/010:08:250
759026992119,1cyclictest0-21swapper/008:07:270
759026992113,6cyclictest787969-21sed08:30:010
139099210,14rtkit-daemon0-21swapper/211:27:202
8665672200,4sleep13056-21PK-Backend12:10:151
759041992018,1cyclictest0-21swapper/312:33:563
759041992018,1cyclictest0-21swapper/309:37:413
759041992018,1cyclictest0-21swapper/308:00:203
759041992017,2cyclictest3056-21PK-Backend08:33:593
759041992017,2cyclictest3056-21PK-Backend07:52:573
759041992017,2cyclictest3056-21PK-Backend07:52:563
759041992012,6cyclictest883-21dbus-daemon11:00:013
759041992012,6cyclictest836487-21ls10:45:123
75904199201,16cyclictest823795-21memory10:10:163
75904199200,1cyclictest0-21swapper/309:04:013
75904199200,19cyclictest0-21swapper/312:29:373
75904199200,19cyclictest0-21swapper/311:58:543
75904199200,19cyclictest0-21swapper/311:43:483
75904199200,19cyclictest0-21swapper/311:20:093
75904199200,19cyclictest0-21swapper/310:20:053
75904199200,19cyclictest0-21swapper/309:28:483
75904199200,19cyclictest0-21swapper/309:10:173
75904199200,19cyclictest0-21swapper/307:47:023
75903499202,16cyclictest1095-21runrttasks12:30:002
759034992018,1cyclictest0-21swapper/211:50:072
759034992018,1cyclictest0-21swapper/209:28:382
759034992018,1cyclictest0-21swapper/209:15:212
759034992018,1cyclictest0-21swapper/209:02:492
759034992018,1cyclictest0-21swapper/207:34:012
75903499201,13cyclictest856996-21apt-get11:45:032
75903499200,1cyclictest0-21swapper/211:02:412
75903499200,1cyclictest0-21swapper/208:55:462
75903499200,1cyclictest0-21swapper/208:43:282
75903499200,1cyclictest0-21swapper/208:00:102
75903499200,19cyclictest0-21swapper/212:20:152
75903499200,19cyclictest0-21swapper/211:44:042
75903499200,19cyclictest0-21swapper/211:35:252
75903499200,19cyclictest0-21swapper/210:57:062
75903499200,19cyclictest0-21swapper/209:46:162
759029992018,1cyclictest3056-21PK-Backend12:16:271
759029992018,1cyclictest3056-21PK-Backend11:57:551
759029992018,1cyclictest3056-21PK-Backend09:26:211
759029992018,1cyclictest3056-21PK-Backend07:20:541
759029992018,1cyclictest0-21swapper/109:55:411
759029992018,1cyclictest0-21swapper/108:48:551
759029992017,2cyclictest3056-21PK-Backend11:02:461
759029992015,3cyclictest771206-21timerwakeupswit07:40:231
759029992013,6cyclictest789026-21sed08:30:221
75902999200,1cyclictest3056-21PK-Backend11:16:451
75902999200,19cyclictest0-21swapper/108:58:231
75902999200,19cyclictest0-21swapper/108:00:431
759026992018,1cyclictest0-21swapper/011:22:470
759026992018,1cyclictest0-21swapper/010:00:350
759026992018,1cyclictest0-21swapper/009:26:550
759026992018,1cyclictest0-21swapper/008:44:170
759026992018,1cyclictest0-21swapper/008:02:390
759026992018,1cyclictest0-21swapper/007:18:120
759026992015,4cyclictest873393-21/usr/sbin/munin12:30:090
75902699200,6cyclictest864183-21apt12:05:000
75902699200,19cyclictest0-21swapper/011:46:330
75902699200,19cyclictest0-21swapper/007:20:350
139099200,3rtkit-daemon0-21swapper/210:13:272
139099200,14rtkit-daemon3056-21PK-Backend10:42:062
139099200,14rtkit-daemon3056-21PK-Backend10:25:262
8369382190,4sleep13056-21PK-Backend10:45:181
75904199194,13cyclictest0-21swapper/309:50:163
759041991917,1cyclictest3056-21PK-Backend08:21:363
759041991917,1cyclictest0-21swapper/312:00:093
759041991917,1cyclictest0-21swapper/311:31:033
759041991917,1cyclictest0-21swapper/310:15:123
759041991917,1cyclictest0-21swapper/309:05:513
759041991917,1cyclictest0-21swapper/307:40:193
759041991917,1cyclictest0-21swapper/307:21:073
759041991917,1cyclictest0-21swapper/307:19:183
759041991917,1cyclictest0-21swapper/307:11:193
759041991912,5cyclictest436-20systemd-journal09:59:553
759041991910,7cyclictest819755-21apt10:00:133
75904199190,7cyclictest864458-21apt12:05:093
75904199190,1cyclictest3056-21PK-Backend08:06:563
75904199190,1cyclictest3056-21PK-Backend07:34:383
75904199190,1cyclictest0-21swapper/311:17:513
75904199190,1cyclictest0-21swapper/310:27:503
75904199190,1cyclictest0-21swapper/310:06:143
75904199190,1cyclictest0-21swapper/309:40:313
75904199190,1cyclictest0-21swapper/309:24:223
75904199190,18cyclictest0-21swapper/312:13:213
75904199190,18cyclictest0-21swapper/310:51:153
75904199190,18cyclictest0-21swapper/309:33:143
75904199190,18cyclictest0-21swapper/308:10:213
75904199190,16cyclictest842260-21python311:00:203
759034991917,1cyclictest0-21swapper/210:51:322
759034991917,1cyclictest0-21swapper/208:46:072
759034991917,1cyclictest0-21swapper/208:30:452
759034991912,4cyclictest862596-21latency_hist12:00:002
75903499190,1cyclictest0-21swapper/212:36:162
759029991917,1cyclictest3056-21PK-Backend11:10:241
759029991917,1cyclictest3056-21PK-Backend10:31:301
759029991917,1cyclictest3056-21PK-Backend09:34:591
759029991917,1cyclictest0-21swapper/110:25:481
759029991917,1cyclictest0-21swapper/108:21:581
759029991917,1cyclictest0-21swapper/107:39:071
75902999190,1cyclictest3056-21PK-Backend09:44:041
75902999190,1cyclictest0-21swapper/108:37:331
75902999190,1cyclictest0-21swapper/108:27:131
75902699196,6cyclictest141rcu_preempt10:50:200
759026991917,1cyclictest0-21swapper/012:27:020
759026991917,1cyclictest0-21swapper/011:15:180
759026991917,1cyclictest0-21swapper/009:22:080
759026991917,1cyclictest0-21swapper/008:23:000
759026991917,1cyclictest0-21swapper/007:58:400
759026991917,1cyclictest0-21swapper/007:40:150
759026991914,3cyclictest1095-21runrttasks11:05:220
759026991913,5cyclictest816019-21perf09:50:000
759026991913,5cyclictest775230-21cut07:55:010
759026991913,5cyclictest775230-21cut07:55:000
759026991912,5cyclictest850565-21uname11:25:160
759026991912,5cyclictest829977-21dpkg10:30:020
75902699190,1cyclictest0-21swapper/012:13:520
75902699190,1cyclictest0-21swapper/010:37:390
75902699190,1cyclictest0-21swapper/010:30:430
75902699190,18cyclictest0-21swapper/011:44:050
75902699190,18cyclictest0-21swapper/009:09:590
75902699190,18cyclictest0-21swapper/008:51:260
75902699190,18cyclictest0-21swapper/007:39:130
8188852180,3chrt0-21swapper/009:55:210
759041991816,1cyclictest0-21swapper/309:19:083
759041991813,3cyclictest868184-21apt12:15:133
759041991811,5cyclictest854221-21basename11:35:153
759041991811,5cyclictest815103-21ls09:45:173
75903499189,6cyclictest837374-21sshd10:46:282
759034991816,1cyclictest0-21swapper/211:21:262
759034991812,5cyclictest852483-21ls11:30:152
759029991816,1cyclictest3056-21PK-Backend10:09:191
759029991815,2cyclictest3056-21PK-Backend10:51:511
759029991810,6cyclictest794679-21apt-get08:50:031
75902999180,1cyclictest3056-21PK-Backend10:20:551
759026991816,1cyclictest0-21swapper/009:39:400
759026991816,1cyclictest0-21swapper/007:34:000
759026991813,3cyclictest803616-21idleruntime-cro09:15:000
759026991812,4cyclictest797332-21forks08:55:150
759026991812,4cyclictest794953-21date08:50:000
75902699180,1cyclictest0-21swapper/009:03:380
759041991715,1cyclictest3056-21PK-Backend08:35:363
759041991715,1cyclictest3056-21PK-Backend07:29:113
759041991715,1cyclictest0-21swapper/308:55:223
759041991710,5cyclictest42-21ksoftirqd/311:45:153
759041991710,5cyclictest42-21ksoftirqd/311:05:123
75904199170,15cyclictest875507-21diskstats12:35:153
759034991712,3cyclictest787781-21idleruntime-cro08:30:002
75902999178,7cyclictest775732-21df07:55:131
759029991715,1cyclictest3056-21PK-Backend10:59:281
759029991710,5cyclictest766908-21cat07:30:121
75902699172,3cyclictest141rcu_preempt12:40:000
759026991715,1cyclictest0-21swapper/012:09:110
759026991715,1cyclictest0-21swapper/010:41:440
759026991715,1cyclictest0-21swapper/008:38:090
759026991712,3cyclictest825617-21needreboot10:15:160
759026991711,4cyclictest781267-21awk08:10:140
759026991710,5cyclictest788194-21expr08:30:120
75902699169,5cyclictest827571-21grep10:20:240
759026991614,1cyclictest0-21swapper/012:22:230
759026991614,1cyclictest0-21swapper/011:34:570
759026991614,1cyclictest0-21swapper/010:57:120
759026991613,2cyclictest860715-21cron11:55:000
759026991610,5cyclictest812959-21tr09:40:120
8505662150,4sleep13056-21PK-Backend11:25:161
759041991510,3cyclictest2156-21gnome-shell07:55:093
75902999158,5cyclictest856324-21proc_pri11:40:171
75902999152,5cyclictest854664-21python311:35:201
759029991513,1cyclictest3056-21PK-Backend09:08:011
75902999150,4cyclictest3056-21PK-Backend12:29:561
759026991513,1cyclictest0-21swapper/011:35:590
759026991513,1cyclictest0-21swapper/011:00:030
759026991513,1cyclictest0-21swapper/009:52:440
759026991511,3cyclictest845038-21tr11:10:110
759026991511,2cyclictest823370-21tr10:10:110
75902699150,14cyclictest0-21swapper/009:30:550
75904199146,6cyclictest42-21ksoftirqd/308:40:123
75903499141,10cyclictest775117-21munin-run07:55:002
75903499141,10cyclictest775117-21munin-run07:54:592
75902699149,3cyclictest836406-21cstates10:45:120
75902699147,5cyclictest759388-21expr07:10:110
75902699146,6cyclictest13-21ksoftirqd/009:18:270
759026991412,1cyclictest0-21swapper/012:17:430
759041991311,1cyclictest3056-21PK-Backend08:26:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional