You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-24 - 12:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sun May 24, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
108172240165,24sleep10-21swapper/119:09:451
105642203165,25sleep30-21swapper/319:06:303
88702202165,24sleep20-21swapper/219:05:032
107242202164,26sleep00-21swapper/019:08:320
3128521750,4sleep31093899cyclictest21:05:253
3128521750,4sleep31093899cyclictest21:05:253
2118121450,6sleep21093499cyclictest20:45:012
1093499103100,2cyclictest0-21swapper/222:29:112
151012630,3sleep01092299cyclictest22:55:250
304472610,3sleep20-21swapper/221:05:162
304472610,3sleep20-21swapper/221:05:162
194742560,2sleep10-21swapper/120:40:151
147772440,4sleep121-21rcuc/122:55:231
1093899421,32cyclictest0-21swapper/300:35:213
10938994127,6cyclictest0-21swapper/321:04:113
1093499411,29cyclictest0-21swapper/220:35:032
1093899401,14cyclictest0-21swapper/319:15:043
10938993923,6cyclictest0-21swapper/320:11:103
1092899390,13cyclictest0-21swapper/122:40:171
10938993810,11cyclictest0-21swapper/321:30:273
10938993726,3cyclictest0-21swapper/320:28:343
1093499371,26cyclictest0-21swapper/221:56:302
1092299377,14cyclictest24777-21munin-run19:40:010
76832360,4sleep21093499cyclictest22:40:152
10938993617,8cyclictest7269-21uname21:25:193
1093899361,27cyclictest13071-21vmstat22:50:263
10934993633,2cyclictest0-21swapper/221:40:112
1093499361,24cyclictest0-21swapper/220:50:312
10938993523,3cyclictest0-21swapper/320:57:193
10938993517,11cyclictest0-21swapper/319:29:433
1093899351,27cyclictest0-21swapper/319:55:433
1093899351,27cyclictest0-21swapper/319:55:423
1093899351,23cyclictest0-21swapper/322:21:503
10938993512,10cyclictest26072-21ntp_states19:40:233
10934993532,2cyclictest0-21swapper/222:52:192
1093899349,10cyclictest0-21swapper/320:43:103
10938993412,9cyclictest14013-21df_inode22:55:153
1093899341,26cyclictest0-21swapper/323:51:153
1093899341,20cyclictest0-21swapper/320:47:233
1093899341,20cyclictest0-21swapper/320:08:583
1093899341,13cyclictest0-21swapper/322:17:143
1093499341,25cyclictest0-21swapper/220:48:072
1093499341,23cyclictest0-21swapper/220:01:582
1092899341,8cyclictest0-21swapper/100:05:541
108892340,4sleep011-21rcuc/021:35:120
10938993313,11cyclictest0-21swapper/322:05:023
1093899331,26cyclictest0-21swapper/320:37:543
1093899331,24cyclictest0-21swapper/319:45:073
1093899331,23cyclictest0-21swapper/319:14:423
1093899331,21cyclictest0-21swapper/320:34:023
1093899330,26cyclictest0-21swapper/322:49:263
1092299336,12cyclictest10634-21perf21:35:020
10938993213,10cyclictest0-21swapper/323:46:553
10938993212,8cyclictest0-21swapper/320:23:303
1093899321,24cyclictest0-21swapper/321:49:583
10938993212,3cyclictest0-21swapper/320:52:113
1093899321,22cyclictest0-21swapper/322:12:583
1093899321,13cyclictest0-21swapper/323:43:433
1093899320,26cyclictest0-21swapper/323:58:273
1093899320,26cyclictest0-21swapper/321:50:463
1093899320,24cyclictest0-21swapper/300:06:033
1092899321,24cyclictest0-21swapper/119:25:281
1092299327,12cyclictest18483-21munin-run00:20:010
301252310,4sleep21093499cyclictest23:30:242
10938993115,11cyclictest0-21swapper/300:19:173
1093899311,22cyclictest0-21swapper/321:16:433
1093899311,22cyclictest0-21swapper/319:53:113
1093899311,21cyclictest839-21lldpd23:31:033
1093899311,18cyclictest0-21swapper/300:03:133
1093899311,17cyclictest0-21swapper/321:43:273
1093899311,14cyclictest0-21swapper/322:39:143
1093899311,13cyclictest0-21swapper/319:37:153
1093899311,13cyclictest0-21swapper/319:33:073
1093899310,23cyclictest0-21swapper/323:05:153
10934993128,2cyclictest0-21swapper/223:44:122
1092899312,14cyclictest16365-40dnf20:32:091
1093899301,23cyclictest0-21swapper/323:25:473
1093899301,23cyclictest0-21swapper/322:04:503
1093899301,20cyclictest0-21swapper/322:40:223
1093899301,15cyclictest0-21swapper/300:32:313
1093899301,15cyclictest0-21swapper/300:32:313
1093899301,15cyclictest0-21swapper/300:21:153
1093899301,14cyclictest0-21swapper/320:19:303
1093899301,13cyclictest0-21swapper/323:20:513
1093899301,13cyclictest0-21swapper/323:20:513
1093899301,11cyclictest23165-21grep23:15:193
1093899301,11cyclictest0-21swapper/323:03:433
10938993010,8cyclictest0-21swapper/322:25:063
1093499301,12cyclictest0-21swapper/220:05:182
1093499300,23cyclictest0-21swapper/222:19:432
1093499300,16cyclictest0-21swapper/223:59:382
1093499300,13cyclictest0-21swapper/221:46:302
1092899301,23cyclictest0-21swapper/122:11:131
1093899291,15cyclictest0-21swapper/321:20:033
1093899291,13cyclictest0-21swapper/321:56:183
1093899291,13cyclictest0-21swapper/321:11:473
1093899290,11cyclictest0-21swapper/319:21:393
1093499291,13cyclictest0-21swapper/222:01:072
1093499291,11cyclictest0-21swapper/219:56:062
1093499291,11cyclictest0-21swapper/219:56:062
1093499290,23cyclictest0-21swapper/222:34:232
1093499290,17cyclictest0-21swapper/200:35:342
1092899291,9cyclictest0-21swapper/123:41:081
1092899291,19cyclictest18503-21ntp_offset21:50:211
1092899291,11cyclictest0-21swapper/121:25:001
1092899291,10cyclictest0-21swapper/100:25:421
1093899289,11cyclictest0-21swapper/322:34:023
1093899281,13cyclictest0-21swapper/321:36:193
1093899281,12cyclictest0-21swapper/300:10:553
1093899280,13cyclictest0-21swapper/323:36:273
1093499289,11cyclictest0-21swapper/219:16:252
1093499281,12cyclictest0-21swapper/223:11:112
1093499281,12cyclictest0-21swapper/221:25:082
1093499281,12cyclictest0-21swapper/219:10:062
1093499281,11cyclictest0-21swapper/221:15:142
1093499280,11cyclictest0-21swapper/223:00:592
1092899286,3cyclictest0-21swapper/119:14:091
1092899281,9cyclictest0-21swapper/123:16:371
1092899281,9cyclictest0-21swapper/122:21:531
1092899281,26cyclictest0-21swapper/121:46:081
1092899281,14cyclictest18232-21timerwakeupswit20:35:251
1092299285,11cyclictest9676-21perf00:00:020
1093899271,13cyclictest0-21swapper/320:00:503
1093899271,13cyclictest0-21swapper/300:25:473
1093899271,11cyclictest0-21swapper/323:10:553
1093499278,9cyclictest0-21swapper/200:21:402
1093499271,11cyclictest0-21swapper/219:31:512
1092899277,3cyclictest0-21swapper/120:45:531
1092899271,25cyclictest0-21swapper/122:02:171
1092899271,24cyclictest0-21swapper/119:19:011
1092899271,17cyclictest0-21swapper/123:10:221
1092899271,11cyclictest11061-21expr21:35:131
1092899271,11cyclictest0-21swapper/123:31:121
1093499261,9cyclictest0-21swapper/220:57:352
1093499261,11cyclictest0-21swapper/222:05:152
1093499260,12cyclictest0-21swapper/219:39:462
1093499260,10cyclictest0-21swapper/221:35:262
1092899269,4cyclictest32572-21diskstats21:10:131
1092899269,3cyclictest5265-21munin-run23:50:011
1092899267,3cyclictest0-21swapper/121:05:281
1092899267,3cyclictest0-21swapper/121:05:281
1092899261,24cyclictest0-21swapper/120:08:451
1092899261,24cyclictest0-21swapper/119:58:441
1092899261,24cyclictest0-21swapper/119:58:441
1092899261,22cyclictest22283-40dnf19:31:221
1092899261,13cyclictest0-21swapper/121:03:001
1092899261,12cyclictest0-21swapper/100:15:301
1093499251,12cyclictest0-21swapper/200:13:302
1093499251,10cyclictest0-21swapper/222:55:232
1093499251,10cyclictest0-21swapper/200:32:022
1093499251,10cyclictest0-21swapper/200:32:022
1093499250,10cyclictest827-21runrttasks20:30:112
1092899258,6cyclictest0-21swapper/119:47:121
1092899251,15cyclictest0-21swapper/100:34:121
1092899251,15cyclictest0-21swapper/100:34:121
1092899251,12cyclictest9615-21df_inode22:45:161
1092899251,12cyclictest0-21swapper/121:30:201
1092899251,11cyclictest14563-21cat20:30:021
1092899251,11cyclictest10565-21hddtemp_smartct00:00:161
1092899251,10cyclictest0-21swapper/120:24:411
1092899250,23cyclictest0-21swapper/122:09:501
1092899250,23cyclictest0-21swapper/100:20:501
1092899250,16cyclictest0-21swapper/120:14:511
1093499241,14cyclictest0-21swapper/220:27:112
1093499241,10cyclictest0-21swapper/219:54:542
1093499241,10cyclictest0-21swapper/219:41:342
1093499240,9cyclictest0-21swapper/200:08:222
1093499240,10cyclictest0-21swapper/223:50:592
1093499240,10cyclictest0-21swapper/223:36:552
1093499240,10cyclictest0-21swapper/223:16:592
1093499240,10cyclictest0-21swapper/220:21:582
1092899249,3cyclictest0-21swapper/119:23:371
1092899241,3cyclictest21670-21latency_hist22:00:011
1092899241,3cyclictest0-21swapper/119:52:501
1092899241,14cyclictest0-21swapper/123:54:551
1092899241,13cyclictest29684-21sendmail_mailst22:15:251
1092899241,11cyclictest27209-21diskstats23:25:131
1092899241,11cyclictest0-21swapper/119:41:121
1092899241,10cyclictest0-21swapper/119:36:381
10928992410,5cyclictest31816-21fschecks_time23:35:151
1092299241,12cyclictest31992-21diskmemload23:50:020
1093499238,9cyclictest0-21swapper/223:07:372
1093499237,3cyclictest0-21swapper/219:28:252
1093499234,10cyclictest0-21swapper/219:48:402
1093499231,8cyclictest0-21swapper/221:21:542
1093499231,10cyclictest0-21swapper/220:12:182
1092899238,3cyclictest16021-21tr23:00:131
1092899237,3cyclictest0-21swapper/120:56:331
1092899237,14cyclictest6936-21cat22:40:021
1092899236,7cyclictest0-21swapper/121:27:501
1092899231,3cyclictest13420-21fschecks_count21:40:151
10928992312,3cyclictest0-21swapper/100:10:521
1092899231,10cyclictest0-21swapper/122:50:471
1092299231,11cyclictest21010-21awk20:45:000
1092299230,11cyclictest26032-21awk22:10:010
1093499228,7cyclictest0-21swapper/200:01:002
1093499227,3cyclictest28880-21ntp_states21:00:212
1093499227,3cyclictest0-21swapper/223:26:272
10934992210,2cyclictest0-21swapper/220:17:262
1093499220,9cyclictest0-21swapper/222:13:312
1093499220,10cyclictest0-21swapper/221:33:382
10928992212,3cyclictest0-21swapper/100:39:201
1092899221,12cyclictest0-21swapper/120:52:051
1092899221,10cyclictest8131-21sort20:15:141
1092899221,10cyclictest8021-21cstates23:55:121
1092299227,6cyclictest0-21swapper/023:39:340
1092299227,6cyclictest0-21swapper/022:04:590
1092299225,15cyclictest25093-21basename00:35:010
1092299225,15cyclictest25093-21basename00:35:010
1093499218,3cyclictest0-21swapper/222:36:452
1093499217,3cyclictest0-21swapper/223:22:112
1093499217,3cyclictest0-21swapper/223:22:112
1093499215,3cyclictest0-21swapper/221:51:102
1093499211,9cyclictest0-21swapper/200:15:342
1093499210,9cyclictest0-21swapper/223:49:352
1092899216,3cyclictest0-21swapper/123:23:291
1092899216,3cyclictest0-21swapper/123:23:281
1092899211,11cyclictest1545-21grep20:00:131
1092299215,14cyclictest0-21swapper/021:05:000
1092299212,16cyclictest25408-21idleruntime-cro20:55:000
1092299211,12cyclictest8107-21sshd21:26:070
1092299211,11cyclictest0-21swapper/021:56:280
1092299211,11cyclictest0-21swapper/020:35:180
1093499207,3cyclictest0-21swapper/222:20:332
1093499205,8cyclictest0-21swapper/200:29:442
1093499201,10cyclictest0-21swapper/219:24:272
1092899204,3cyclictest0-21swapper/121:18:441
1092899200,5cyclictest0-21swapper/122:31:501
1092899200,14cyclictest0-21swapper/123:06:561
1092299209,3cyclictest370-21/usr/sbin/munin22:25:260
1092299209,2cyclictest25050-21fschecks_count23:20:130
1092299209,2cyclictest25050-21fschecks_count23:20:130
1092299208,3cyclictest22465-21sed00:25:270
1092299207,5cyclictest0-21swapper/000:11:150
1092299207,3cyclictest0-21swapper/022:46:430
1092299202,10cyclictest2063-21/usr/sbin/munin21:15:250
1092299201,11cyclictest0-21swapper/021:49:400
1092299201,10cyclictest8640-21head20:15:180
1092299201,10cyclictest7493-21sshd20:11:080
1092299201,10cyclictest19007-21unixbench_singl21:50:260
1093499194,2cyclictest0-21swapper/222:45:152
1093499191,11cyclictest0-21swapper/221:13:432
1092899190,10cyclictest0-21swapper/122:25:131
1092299197,4cyclictest26813-21grep00:35:270
1092299197,4cyclictest0-21swapper/000:00:140
1092299197,3cyclictest22119-21unixbench_multi19:30:250
1092299195,2cyclictest87350irq/126-enp1s0-22:12:570
1092299194,6cyclictest27202-21/usr/sbin/munin19:45:180
1092299194,13cyclictest11-21rcuc/023:35:000
1092299193,7cyclictest0-21swapper/021:41:290
1092299192,9cyclictest11-21rcuc/020:45:260
1092299192,4cyclictest27925-21ntp_states23:25:200
1092299192,4cyclictest27588-21munin-run21:00:000
1092299191,5cyclictest0-21swapper/023:41:020
1092299191,4cyclictest0-21swapper/022:15:550
1092299191,15cyclictest5768-21tail21:20:250
1092299191,12cyclictest0-21swapper/019:40:240
1092299191,10cyclictest8486-21timerwakeupswit22:40:240
1092299191,10cyclictest3642-21/usr/sbin/munin20:05:210
1092299191,10cyclictest22636-21cut23:15:130
1092299191,10cyclictest20601-21sshd00:24:080
1092299191,10cyclictest20275-21/usr/sbin/munin23:10:270
1092299190,12cyclictest0-21swapper/021:05:160
1092299190,12cyclictest0-21swapper/021:05:160
41802180,3sleep04185-21unixbench_multi22:30:250
1092299186,5cyclictest0-21swapper/022:50:520
1092299182,15cyclictest0-21swapper/000:08:460
1092299182,11cyclictest0-21swapper/022:20:220
1092299181,9cyclictest0-21swapper/020:23:070
1092299181,11cyclictest0-21swapper/023:05:010
1092299177,8cyclictest1-21systemd20:00:010
1092299177,8cyclictest1-21systemd20:00:010
1092299175,5cyclictest0-21swapper/023:54:590
1092299175,3cyclictest0-21swapper/021:10:390
1092299174,4cyclictest579-21systemd-journal20:05:010
1092299171,4cyclictest20383-21munin-run19:30:020
10922991713,3cyclictest0-21swapper/019:22:260
1092299171,13cyclictest31992-21diskmemload22:35:170
10922991710,4cyclictest12694-21grep19:10:250
1092299170,5cyclictest0-21swapper/023:05:310
1092299163,11cyclictest11-21rcuc/019:15:160
1092299161,5cyclictest14816-21cstates20:30:120
1092299157,3cyclictest87550irq/128-enp1s0-19:50:070
10922991511,3cyclictest0-21swapper/020:27:580
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional