You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-02 - 09:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3s.osadl.org (updated Tue Jun 02, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6872239161,24sleep00-21swapper/019:09:390
5662237163,23sleep20-21swapper/219:08:112
4752222153,57sleep30-21swapper/319:07:043
3512204167,24sleep10-21swapper/119:05:331
1351221870,4sleep385699cyclictest19:35:183
2575021410,4sleep285299cyclictest22:30:172
104222670,1sleep10-21swapper/119:30:011
210122590,3sleep083699cyclictest23:35:000
171712500,4sleep3401ktimersoftd/300:39:213
139142490,1sleep30-21swapper/320:50:143
856993927,6cyclictest0-21swapper/323:28:413
852993913,23cyclictest25963-21/usr/sbin/munin20:05:182
85299381,28cyclictest0-21swapper/223:20:312
83699389,13cyclictest14758-21basename19:40:010
856993726,3cyclictest0-21swapper/319:20:013
85699371,21cyclictest0-21swapper/321:23:093
83699378,13cyclictest13794-21munin-run22:05:010
83699376,15cyclictest18802-21munin-run23:30:000
85699351,15cyclictest0-21swapper/321:27:173
85299355,4cyclictest23877-40dnf22:25:252
85299351,5cyclictest21221-21/usr/sbin/munin23:35:182
85299351,5cyclictest21221-21/usr/sbin/munin23:35:182
85299351,27cyclictest0-21swapper/200:25:112
856993417,8cyclictest0-21swapper/300:02:493
85699341,21cyclictest14828-21irqstats22:05:173
85699341,21cyclictest0-21swapper/319:11:083
852993413,18cyclictest3980-21/usr/sbin/munin19:15:232
852993410,3cyclictest8053-21smart_sda20:35:252
85299334,3cyclictest87350irq/126-enp1s0-21:20:022
85299331,5cyclictest736-21/usr/sbin/munin21:35:232
85299331,5cyclictest30013-21/usr/sbin/munin23:55:272
852993310,4cyclictest0-21swapper/223:30:162
852993310,2cyclictest15589-21/usr/sbin/munin20:55:182
83699338,12cyclictest17248-21awk00:40:010
85699321,25cyclictest0-21swapper/323:42:333
85699320,26cyclictest0-21swapper/321:38:113
85299329,3cyclictest30978-21/usr/sbin/munin21:30:242
85299328,3cyclictest8988-21/usr/sbin/munin20:40:152
85299327,4cyclictest19812-21/usr/sbin/munin22:15:252
85299321,16cyclictest0-21swapper/221:57:342
85299321,15cyclictest0-21swapper/221:48:512
85299321,14cyclictest0-21swapper/220:01:552
852993211,3cyclictest27901-21cat21:20:252
852993211,2cyclictest2073-21/usr/sbin/munin00:05:162
852993210,2cyclictest30357-21/usr/sbin/munin20:15:232
852993210,19cyclictest8671-21/usr/sbin/munin00:20:242
85699311,23cyclictest0-21swapper/300:26:473
85699311,21cyclictest0-21swapper/320:56:213
85699311,12cyclictest0-21swapper/322:53:443
85699310,24cyclictest0-21swapper/319:59:453
85699310,21cyclictest0-21swapper/300:30:133
85299318,4cyclictest7259-21timerwakeupswit23:00:242
85299317,3cyclictest0-21swapper/222:05:252
85299315,3cyclictest87350irq/126-enp1s0-20:20:122
85299313,3cyclictest87350irq/126-enp1s0-20:30:232
85299311,21cyclictest0-21swapper/222:44:592
852993112,16cyclictest7538-21ntp_states00:15:222
85299311,20cyclictest0-21swapper/219:59:582
85299311,19cyclictest0-21swapper/222:37:142
85299311,19cyclictest0-21swapper/220:26:432
85299311,13cyclictest0-21swapper/222:48:552
85299311,12cyclictest0-21swapper/223:25:352
852993110,3cyclictest25996-21cut23:45:172
85299310,20cyclictest0-21swapper/220:54:352
83699313,13cyclictest12858-21perf00:30:010
85699301,26cyclictest0-21swapper/300:14:273
85699301,19cyclictest0-21swapper/323:34:253
85699301,13cyclictest0-21swapper/323:52:453
85699301,13cyclictest0-21swapper/322:36:173
85699301,13cyclictest0-21swapper/321:40:293
85699301,12cyclictest0-21swapper/300:23:093
85299309,4cyclictest29421-21smart_sda20:10:242
85299307,19cyclictest0-21swapper/222:10:142
85299306,4cyclictest14473-21/usr/sbin/munin00:30:262
85299306,4cyclictest1070-21/usr/sbin/munin19:10:262
85299301,14cyclictest0-21swapper/221:26:432
85299301,12cyclictest0-21swapper/200:11:152
852993010,3cyclictest29331-21sed23:50:262
85699291,19cyclictest0-21swapper/323:05:243
85699291,19cyclictest0-21swapper/320:28:003
85699291,17cyclictest0-21swapper/319:26:353
85699291,12cyclictest0-21swapper/322:48:053
85699291,12cyclictest0-21swapper/300:18:453
85299298,3cyclictest21905-21cat22:20:242
85299298,2cyclictest827-21runrttasks22:04:022
85299298,2cyclictest20991-21netstat21:05:212
85299297,3cyclictest5807-21cat23:00:022
85299295,5cyclictest0-21swapper/223:10:172
85299292,3cyclictest87350irq/126-enp1s0-23:05:192
85299291,19cyclictest0-21swapper/221:53:022
85299291,12cyclictest0-21swapper/223:18:392
852992910,3cyclictest827-21runrttasks19:49:462
83699294,12cyclictest501-21awk21:35:010
85699281,12cyclictest0-21swapper/322:23:293
85699281,12cyclictest0-21swapper/322:16:373
85699281,11cyclictest0-21swapper/320:30:533
85699280,16cyclictest0-21swapper/322:04:133
85299289,3cyclictest1973-21fschecks_time22:50:162
85299289,2cyclictest6356-21cstates19:20:122
85299288,3cyclictest827-21runrttasks19:32:002
85299288,3cyclictest19985-21gltestperf19:50:162
85299287,3cyclictest196422chrt21:03:372
85299282,9cyclictest14281-21tail19:35:252
85299281,5cyclictest0-21swapper/219:25:112
852992810,3cyclictest3370-21dnf21:40:152
852992810,3cyclictest0-21swapper/200:00:192
83699289,16cyclictest31999-21awk00:00:000
85699271,24cyclictest0-21swapper/319:44:053
85699271,18cyclictest0-21swapper/323:45:533
85699271,13cyclictest0-21swapper/321:48:493
85699271,12cyclictest0-21swapper/322:33:533
85699271,12cyclictest0-21swapper/320:47:333
85699271,12cyclictest0-21swapper/320:24:223
85699271,11cyclictest3102-21kworker/u8:119:45:453
85699271,11cyclictest0-21swapper/322:27:333
85699271,10cyclictest0-21swapper/322:58:003
85299278,3cyclictest11589-21grep20:45:162
85299271,12cyclictest0-21swapper/221:10:112
85299271,12cyclictest0-21swapper/221:10:112
83699271,13cyclictest16956-21perf19:45:010
83699270,13cyclictest6266-21perf00:15:020
85699264,3cyclictest0-21swapper/319:50:413
85699261,23cyclictest0-21swapper/322:12:373
85699261,23cyclictest0-21swapper/321:07:533
85699261,12cyclictest13698-21smart_sda23:15:253
85699261,12cyclictest0-21swapper/321:17:113
85699261,11cyclictest0-21swapper/321:50:173
85699260,19cyclictest0-21swapper/320:07:583
85699260,16cyclictest0-21swapper/320:35:033
85699260,15cyclictest0-21swapper/323:59:393
85299261,4cyclictest0-21swapper/223:40:122
85299261,3cyclictest15613-21diskstats00:35:142
85299261,10cyclictest0-21swapper/219:40:022
83699262,15cyclictest9241-21kworker/0:022:05:390
85699252,11cyclictest8011-40dnf19:23:273
85699251,13cyclictest0-21swapper/323:14:363
85699251,12cyclictest0-21swapper/320:43:093
85699251,12cyclictest0-21swapper/300:09:293
84799258,9cyclictest0-21swapper/123:02:101
83699252,9cyclictest2694-21kworker/0:321:49:330
83699251,9cyclictest32570-21kworker/0:223:04:250
253362250,4sleep07-21ksoftirqd/020:00:270
85699241,9cyclictest0-21swapper/320:16:483
85699241,17cyclictest0-21swapper/322:43:393
85699241,12cyclictest0-21swapper/323:36:013
85699241,12cyclictest0-21swapper/323:36:013
85699240,12cyclictest0-21swapper/320:12:163
85699240,11cyclictest0-21swapper/323:04:223
84799241,5cyclictest32493-21unixbench_multi21:30:231
84799241,17cyclictest0-21swapper/123:29:181
84799240,16cyclictest87650irq/129-enp1s0-23:45:211
83699241,9cyclictest2694-21kworker/0:321:50:290
85699238,9cyclictest0-21swapper/321:56:553
85699231,12cyclictest0-21swapper/323:22:573
84799232,16cyclictest0-21swapper/100:03:231
847992312,3cyclictest16526-21smart_sda00:35:231
84799231,19cyclictest30145-21sleep20:15:011
84799231,16cyclictest0-21swapper/123:18:141
847992310,11cyclictest0-21swapper/100:20:121
83699232,8cyclictest31880-21kworker/0:019:17:240
83699231,9cyclictest26201-21kworker/0:321:22:350
83699231,10cyclictest26369-21perf21:20:010
286212231,3sleep00-21swapper/023:50:180
85699221,9cyclictest0-21swapper/320:01:383
85699221,14cyclictest0-21swapper/321:01:113
84799225,10cyclictest0-21swapper/122:15:301
84799222,15cyclictest0-21swapper/122:25:511
84799221,9cyclictest0-21swapper/121:38:511
84799221,7cyclictest0-21swapper/122:55:281
847992216,4cyclictest0-21swapper/122:37:361
847992216,3cyclictest4010-21grep21:40:191
84799221,4cyclictest20410-21grep19:50:221
84799221,19cyclictest0-21swapper/123:56:061
84799221,16cyclictest0-21swapper/122:42:401
84799221,16cyclictest0-21swapper/121:57:061
84799221,14cyclictest0-21swapper/100:15:141
84799221,12cyclictest3417-21awk22:55:021
84799221,11cyclictest11880-21tr22:00:121
84799221,10cyclictest207812if_err_enp2s021:05:181
83699222,7cyclictest18998-21kworker/0:019:54:130
83699222,11cyclictest5388-21kworker/0:122:59:490
83699222,10cyclictest3843-21kworker/0:000:09:440
83699221,8cyclictest2130-21kworker/0:120:25:290
83699221,10cyclictest24555-21kworker/0:022:42:230
836992210,2cyclictest7684-21kworker/0:023:09:370
85699216,8cyclictest0-21swapper/321:12:433
85699216,8cyclictest0-21swapper/321:12:433
85699211,8cyclictest0-21swapper/321:34:413
85699211,13cyclictest0-21swapper/319:30:453
84799215,4cyclictest0-21swapper/123:42:441
84799212,14cyclictest0-21swapper/123:39:591
84799212,14cyclictest0-21swapper/123:39:591
84799212,14cyclictest0-21swapper/100:11:311
847992114,2cyclictest0-21swapper/100:06:591
84799211,3cyclictest7755-21sh21:50:141
84799211,3cyclictest14435-21taskset19:36:131
847992113,3cyclictest0-21swapper/100:32:241
847992113,2cyclictest0-21swapper/123:10:011
84799211,15cyclictest0-21swapper/123:12:431
84799211,14cyclictest0-21swapper/122:45:261
84799211,11cyclictest6461-21sendmail_mailtr21:45:241
84799211,11cyclictest30808-21cat21:30:011
83699211,8cyclictest12921-21kworker/0:021:06:450
83699211,11cyclictest29773-21kworker/0:020:17:050
83699211,11cyclictest0-21swapper/022:19:150
83699211,10cyclictest20886-21kworker/0:223:43:440
94022200,2sleep00-21swapper/019:25:200
84799202,3cyclictest0-21swapper/120:32:411
84799202,16cyclictest0-21swapper/119:15:501
847992017,2cyclictest0-21swapper/120:57:471
847992016,3cyclictest0-21swapper/119:13:061
847992014,3cyclictest18913-21pmu-power21:00:221
847992013,4cyclictest20540-21grep23:30:261
84799201,14cyclictest0-21swapper/122:08:591
84799201,11cyclictest27919-21grep21:20:251
84799201,11cyclictest26720-21chrt22:31:531
84799201,11cyclictest17526-21cat22:10:231
84799201,10cyclictest7012-21expr20:35:131
83699206,5cyclictest0-21swapper/000:15:360
83699202,10cyclictest736-21/usr/sbin/munin21:35:230
83699201,6cyclictest21247-21kworker/0:219:58:370
83699201,11cyclictest4599-21/usr/sbin/munin20:30:190
83699201,11cyclictest25618-21/usr/sbin/munin23:45:180
83699201,11cyclictest14628-21/usr/sbin/munin23:20:190
83699201,11cyclictest0-21swapper/022:35:500
296102200,4sleep029608-21uname21:25:190
84799194,8cyclictest14159-21if_err_enp1s020:50:161
84799193,4cyclictest0-21swapper/100:28:151
84799192,16cyclictest0-21swapper/122:23:431
84799192,16cyclictest0-21swapper/121:19:591
84799192,16cyclictest0-21swapper/119:24:321
84799191,9cyclictest0-21swapper/119:40:201
84799191,5cyclictest0-21swapper/123:22:571
847991915,3cyclictest0-21swapper/120:16:041
847991915,3cyclictest0-21swapper/119:45:301
84799191,17cyclictest0-21swapper/120:42:341
84799191,17cyclictest0-21swapper/120:01:361
84799191,16cyclictest2174-21munin-run20:25:011
84799191,16cyclictest0-21swapper/121:10:381
84799191,16cyclictest0-21swapper/121:10:381
84799191,10cyclictest28148-21sh23:50:141
83699195,4cyclictest0-21swapper/000:00:270
83699194,5cyclictest15066-21sshd20:51:240
83699191,8cyclictest4346-21kworker/0:020:39:430
83699191,6cyclictest16850-21cut22:10:150
83699191,11cyclictest0-21swapper/021:58:360
83699191,10cyclictest21955-21diskmemload21:10:140
83699191,10cyclictest21955-21diskmemload21:10:140
83699191,10cyclictest21955-21diskmemload00:24:300
83699191,10cyclictest10570-21df23:10:150
83699190,4cyclictest0-21swapper/020:55:250
847991814,3cyclictest0-21swapper/120:47:301
84799181,2cyclictest0-21swapper/120:28:451
84799181,16cyclictest0-21swapper/120:05:561
84799181,16cyclictest0-21swapper/119:56:311
84799181,16cyclictest0-21swapper/119:30:311
83699188,3cyclictest25963-21/usr/sbin/munin20:05:240
83699187,3cyclictest17170-21/usr/sbin/munin19:45:250
83699183,6cyclictest0-21swapper/023:37:300
83699183,6cyclictest0-21swapper/023:37:300
83699181,7cyclictest0-21swapper/023:15:250
83699181,6cyclictest0-21swapper/022:23:370
83699181,3cyclictest25604-21grep22:30:170
83699181,11cyclictest0-21swapper/022:46:310
83699181,10cyclictest21955-21diskmemload21:40:490
83699181,10cyclictest13071-21/usr/sbin/munin00:30:200
83699181,10cyclictest0-21swapper/019:10:120
79342181,2sleep00-21swapper/019:22:270
21172180,3sleep00-21swapper/020:24:280
83699171,8cyclictest32570-21kworker/0:222:50:510
83699171,10cyclictest0-21swapper/020:40:160
83699170,6cyclictest0-21swapper/022:27:590
83699161,6cyclictest17782-21/usr/sbin/munin21:00:110
836991611,3cyclictest29077-21wc20:10:190
83699161,12cyclictest10574-21/usr/sbin/munin19:30:220
836991610,3cyclictest11194-21acpi20:45:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional