You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-14 - 18:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Tue Apr 14, 2026 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
59232243166,24sleep10-21swapper/107:06:551
59232243166,24sleep10-21swapper/107:06:541
60132202165,24sleep30-21swapper/307:08:043
60132202165,24sleep30-21swapper/307:08:043
59242202165,25sleep20-21swapper/207:06:552
59242202165,25sleep20-21swapper/207:06:552
58652201164,25sleep00-21swapper/007:06:090
58652201164,25sleep00-21swapper/007:06:080
10862720,4sleep341-21ksoftirqd/311:50:123
24952420,1sleep30-21swapper/310:40:133
6259993422,3cyclictest0-21swapper/310:04:553
6259993217,9cyclictest0-21swapper/308:35:393
625999311,16cyclictest0-21swapper/307:35:033
625999311,15cyclictest0-21swapper/307:10:393
625999311,14cyclictest0-21swapper/311:26:393
625999311,13cyclictest0-21swapper/311:06:003
625999311,11cyclictest0-21swapper/311:42:313
624099311,22cyclictest87350irq/126-enp1s0-10:12:230
624099311,14cyclictest29521-21munin-run09:15:010
625999301,19cyclictest0-21swapper/308:05:473
625999301,14cyclictest0-21swapper/310:59:563
625999301,14cyclictest0-21swapper/309:49:433
625999301,14cyclictest0-21swapper/308:15:273
625999301,14cyclictest0-21swapper/308:15:273
625999301,13cyclictest0-21swapper/311:30:273
625999301,12cyclictest0-21swapper/312:30:313
625999301,12cyclictest0-21swapper/310:19:593
625999301,11cyclictest27315-21diskmemload12:16:553
625999301,11cyclictest0-21swapper/308:44:043
625999300,14cyclictest0-21swapper/310:53:563
624099309,10cyclictest0-21swapper/008:45:010
6259992914,8cyclictest0-21swapper/307:54:353
625999291,20cyclictest0-21swapper/307:48:033
625999291,13cyclictest0-21swapper/309:51:393
625999291,13cyclictest0-21swapper/309:26:233
625999291,12cyclictest0-21swapper/311:03:483
625999291,12cyclictest0-21swapper/310:06:353
625999291,11cyclictest0-21swapper/309:32:193
625999291,11cyclictest0-21swapper/308:32:533
6259992910,4cyclictest0-21swapper/311:19:353
6259992815,8cyclictest0-21swapper/309:02:273
625999281,14cyclictest0-21swapper/311:39:553
625999281,13cyclictest0-21swapper/311:10:353
625999281,13cyclictest0-21swapper/310:23:113
625999281,12cyclictest0-21swapper/312:03:433
625999281,12cyclictest0-21swapper/310:28:113
625999280,20cyclictest0-21swapper/308:51:403
625999280,19cyclictest0-21swapper/311:48:553
625999280,19cyclictest0-21swapper/308:59:563
625999280,19cyclictest0-21swapper/308:26:113
625999280,15cyclictest0-21swapper/309:44:193
625999280,13cyclictest0-21swapper/312:37:313
624899280,20cyclictest0-21swapper/108:29:081
624099284,11cyclictest20732-21awk08:55:010
625999271,17cyclictest0-21swapper/312:13:553
625999271,12cyclictest0-21swapper/309:14:513
625999271,12cyclictest0-21swapper/308:13:353
625999271,11cyclictest0-21swapper/309:37:113
625999271,11cyclictest0-21swapper/309:21:553
625999270,19cyclictest0-21swapper/307:15:203
625999270,12cyclictest0-21swapper/308:20:153
624099271,5cyclictest0-21swapper/012:35:570
625999269,3cyclictest0-21swapper/307:34:513
625999268,3cyclictest0-21swapper/308:45:323
625999261,13cyclictest0-21swapper/310:33:193
6259992611,3cyclictest0-21swapper/307:44:393
625999261,12cyclictest0-21swapper/311:23:283
625999261,11cyclictest0-21swapper/312:27:313
625999260,13cyclictest0-21swapper/311:55:273
624099262,10cyclictest29262-21kworker/0:109:27:040
624099261,5cyclictest0-21swapper/010:45:220
625999251,12cyclictest0-21swapper/309:16:073
625999251,11cyclictest0-21swapper/309:06:113
625999250,19cyclictest0-21swapper/308:01:033
625999250,10cyclictest0-21swapper/310:46:283
624899255,8cyclictest8108-40dnf07:11:081
624899251,6cyclictest0-21swapper/112:20:201
624899250,5cyclictest0-21swapper/111:35:081
624099251,12cyclictest7207-21perf12:05:020
125062251,4sleep112528-21fschecks_count08:35:151
625999241,12cyclictest0-21swapper/312:05:313
625999241,12cyclictest0-21swapper/310:37:393
625999241,12cyclictest0-21swapper/307:21:043
625999241,11cyclictest0-21swapper/312:23:273
625999240,9cyclictest0-21swapper/310:11:553
625999240,12cyclictest0-21swapper/307:55:033
625399246,9cyclictest87350irq/126-enp1s0-07:20:262
624099241,11cyclictest23772-21/usr/sbin/munin10:15:160
624099241,11cyclictest17798-21/usr/sbin/munin11:15:190
625999231,13cyclictest0-21swapper/307:25:513
6253992316,4cyclictest25316-21grep07:50:152
6248992316,4cyclictest392-21memory09:20:201
624899231,4cyclictest20959-21users10:05:251
624099239,3cyclictest11818-21/usr/sbin/munin12:15:200
624099232,10cyclictest26585-21/usr/sbin/munin11:35:190
624099231,11cyclictest2401-21/usr/sbin/munin10:40:250
624099231,11cyclictest11604-21idleruntime-cro12:15:010
624099230,17cyclictest0-21swapper/010:50:240
625399226,8cyclictest87350irq/126-enp1s0-11:00:192
625399224,5cyclictest4381-21awk10:45:022
625399224,4cyclictest12050irq/123-ahci[0010:12:252
625399221,8cyclictest14705-21diskstats08:40:142
625399221,7cyclictest9396-21perf12:10:012
624899221,12cyclictest13114-21df_inode09:50:141
624899221,11cyclictest0-21swapper/111:27:061
624899221,11cyclictest0-21swapper/110:17:131
624899220,7cyclictest0-21swapper/111:18:591
624099229,3cyclictest87550irq/128-enp1s0-11:50:170
624099229,2cyclictest19373-21/usr/sbin/munin10:05:240
624099226,4cyclictest30268-21fschecks_time09:15:160
624099221,10cyclictest808-21snmpd09:47:180
624099221,10cyclictest808-21snmpd08:02:010
624099220,20cyclictest0-21swapper/009:00:230
625399211,7cyclictest808-21snmpd09:12:122
625399211,6cyclictest19760-21turbostat.cron11:20:012
625399211,6cyclictest18476-21df07:35:142
625399211,17cyclictest30261-21grep11:40:282
625399211,13cyclictest0-21swapper/211:21:462
625399211,11cyclictest6486-21memory08:20:212
625399211,11cyclictest0-21swapper/212:21:192
624899219,3cyclictest16617-21hddtemp_smartct07:30:161
624899215,4cyclictest0-21swapper/111:14:041
624899212,18cyclictest0-21swapper/112:16:361
624899212,17cyclictest0-21swapper/111:49:021
6248992117,3cyclictest0-21swapper/109:35:241
6248992117,3cyclictest0-21swapper/108:44:561
624899211,3cyclictest19925-21sed12:30:271
624899211,2cyclictest9060-21uptime07:15:031
624899211,18cyclictest0-21swapper/109:55:531
624899211,17cyclictest5670-21/usr/sbin/munin10:45:211
624899211,17cyclictest2840-40dnf08:11:231
624899211,17cyclictest13742-21/usr/sbin/munin07:25:151
624899211,14cyclictest0-21swapper/110:00:481
6248992111,3cyclictest145742processes11:05:221
624899211,12cyclictest20319-21grep08:50:261
624899211,10cyclictest0-21swapper/111:34:321
624899211,10cyclictest0-21swapper/110:30:251
624899211,10cyclictest0-21swapper/110:21:021
624099217,3cyclictest808-21snmpd07:26:540
624099217,3cyclictest11546-21/usr/sbin/munin07:20:220
624099216,3cyclictest15941-21/usr/sbin/munin07:30:240
624099211,9cyclictest808-21snmpd09:57:200
68862200,1sleep20-21swapper/210:50:122
625999202,10cyclictest14970-21/usr/sbin/munin09:55:173
625399206,3cyclictest5203-21sshd12:00:092
625399205,4cyclictest0-21swapper/210:05:172
625399202,6cyclictest6180-21/usr/sbin/munin09:35:162
625399201,8cyclictest31724-21awk09:20:002
625399201,7cyclictest579-21systemd-journal11:30:012
625399201,6cyclictest28739-21fschecks_time10:25:172
625399201,6cyclictest0-21swapper/211:58:142
625399201,5cyclictest0-21swapper/210:24:172
625399201,5cyclictest0-21swapper/209:42:552
625399201,10cyclictest1525-21latency_hist09:25:012
625399201,10cyclictest0-21swapper/208:18:182
625399201,10cyclictest0-21swapper/208:18:182
625399200,18cyclictest10971-21perf11:00:012
624899206,4cyclictest0-21swapper/109:26:111
624899204,7cyclictest0-21swapper/111:23:291
624899204,10cyclictest0-21swapper/107:40:141
624899202,16cyclictest0-21swapper/107:35:041
624899201,7cyclictest450-21users08:05:261
624899201,3cyclictest0-21swapper/112:03:081
624899201,18cyclictest0-21swapper/110:55:411
624899201,17cyclictest0-21swapper/111:40:051
624899201,17cyclictest0-21swapper/107:54:441
624899201,13cyclictest0-21swapper/110:12:431
624899201,13cyclictest0-21swapper/109:15:191
624899201,12cyclictest12471-21kworker/u8:210:37:531
624899201,11cyclictest3701-21df08:15:141
624899201,11cyclictest3701-21df08:15:141
624899201,11cyclictest0-21swapper/109:35:011
624899200,4cyclictest0-21swapper/112:25:331
624899200,18cyclictest0-21swapper/109:10:171
624899200,13cyclictest0-21swapper/111:55:551
624099208,3cyclictest0-21swapper/007:15:150
624099206,3cyclictest87350irq/126-enp1s0-08:10:010
624099206,3cyclictest808-21snmpd08:34:070
624099203,14cyclictest17129-21cat12:25:190
624099202,7cyclictest839-21lldpd07:12:000
624099202,15cyclictest3539-21cut11:55:150
624099201,9cyclictest0-21swapper/009:52:190
624099201,8cyclictest819-21lldpd08:36:300
624099201,8cyclictest808-21snmpd11:22:370
624099201,7cyclictest18409-21/usr/sbin/munin12:30:270
624099201,7cyclictest0-21swapper/011:44:390
624099201,6cyclictest0-21swapper/009:30:180
624099201,18cyclictest0-21swapper/011:45:210
624099201,17cyclictest0-21swapper/008:15:240
624099201,17cyclictest0-21swapper/008:15:240
6240992010,3cyclictest87550irq/128-enp1s0-11:05:090
624099200,10cyclictest0-21swapper/009:22:100
625399195,3cyclictest22315-21awk07:45:012
625399195,3cyclictest12050irq/123-ahci[0010:40:022
625399194,9cyclictest0-21swapper/209:30:132
625399194,8cyclictest12471-21kworker/u8:210:33:332
625399194,3cyclictest87350irq/126-enp1s0-11:05:212
625399192,5cyclictest4596-21/usr/sbin/munin10:45:222
625399191,7cyclictest0-21swapper/211:38:572
625399191,7cyclictest0-21swapper/208:25:132
625399191,6cyclictest9916-21ls12:10:112
625399191,6cyclictest0-21swapper/210:02:422
625399191,4cyclictest21094-21tr08:55:132
6253991912,4cyclictest18515-21idleruntime-cro08:50:012
625399191,15cyclictest15900-21sed09:55:192
625399191,15cyclictest14149-21cat08:40:022
625399190,8cyclictest0-21swapper/209:52:072
624899194,4cyclictest0-21swapper/112:11:051
624899192,3cyclictest0-21swapper/108:46:421
624899192,16cyclictest0-21swapper/112:06:311
6248991916,2cyclictest0-21swapper/107:55:271
6248991915,3cyclictest0-21swapper/109:45:321
624899191,17cyclictest0-21swapper/110:51:361
624899191,17cyclictest0-21swapper/110:44:231
624899191,17cyclictest0-21swapper/107:20:511
624899191,16cyclictest0-21swapper/107:45:191
624899191,12cyclictest0-21swapper/110:25:261
624899191,10cyclictest27047-21chrt09:06:411
624899191,10cyclictest11547-21/usr/sbin/munin11:00:141
624899191,10cyclictest0-21swapper/109:05:011
624099193,4cyclictest0-21swapper/008:10:180
624099193,3cyclictest22017-21ntp_states08:55:230
624099192,4cyclictest2173-21munin-run10:40:010
624099192,14cyclictest7763-21/usr/sbin/munin08:25:210
624099191,8cyclictest87350irq/126-enp1s0-07:35:260
624099191,7cyclictest0-21swapper/012:05:160
624099191,6cyclictest28032-21cut10:25:010
624099191,6cyclictest22532-21df_inode11:25:170
624099191,6cyclictest0-21swapper/010:29:080
624099191,5cyclictest1615-1kworker/u9:111:02:470
624099191,3cyclictest24741-21/usr/sbin/munin11:30:150
624099191,3cyclictest10964-21munin-run11:00:010
624099191,10cyclictest87350irq/126-enp1s0-07:57:230
624099190,6cyclictest6496-21kworker/u8:011:10:160
625399183,5cyclictest0-21swapper/212:28:492
625399182,3cyclictest829-21egrep11:50:122
625399181,9cyclictest21291-21hwlatdetect12:35:192
625399181,9cyclictest12401-21gltestperf12:15:162
625399181,6cyclictest29131-21acpi08:00:122
625399181,6cyclictest24366-21gltestperf10:15:162
625399181,6cyclictest0-21swapper/211:46:262
625399181,6cyclictest0-21swapper/211:30:492
625399181,5cyclictest9911-21grep07:15:132
625399181,5cyclictest29056-21cat08:00:012
625399181,5cyclictest27315-21diskmemload09:46:202
625399181,5cyclictest18753-21/usr/sbin/munin08:50:162
625399181,5cyclictest0-21swapper/211:10:262
6253991813,3cyclictest25351-21/usr/sbin/munin09:05:232
6253991813,3cyclictest19466-21ntp_offset12:30:232
625399181,10cyclictest0-21swapper/208:05:172
625399181,10cyclictest0-21swapper/207:47:432
624899188,3cyclictest0-21swapper/108:30:161
6248991815,2cyclictest0-21swapper/111:50:021
6248991815,2cyclictest0-21swapper/109:41:211
6248991814,3cyclictest0-21swapper/112:37:071
624899181,16cyclictest0-21swapper/108:55:491
624899181,16cyclictest0-21swapper/108:20:511
624899181,16cyclictest0-21swapper/108:01:001
624099184,3cyclictest87550irq/128-enp1s0-09:35:150
624099181,5cyclictest24737-21cat07:50:110
624099181,5cyclictest15142-21grep12:20:220
624099181,5cyclictest0-21swapper/010:03:310
624099181,5cyclictest0-21swapper/009:40:180
6240991812,3cyclictest20336-21/usr/sbin/munin07:40:180
624099181,15cyclictest25351-21/usr/sbin/munin09:05:270
624099181,10cyclictest0-21swapper/010:32:260
175282181,2sleep20-21swapper/207:30:272
625399171,6cyclictest0-21swapper/209:25:142
625399171,6cyclictest0-21swapper/209:00:162
625399171,5cyclictest0-21swapper/207:25:062
6253991711,3cyclictest8011-21head07:10:262
6253991710,4cyclictest1538-21fschecks_count08:10:152
624099173,3cyclictest87550irq/128-enp1s0-08:45:120
624099172,3cyclictest23590-21ntp_states07:45:210
625399160,6cyclictest0-21swapper/208:30:522
624099163,3cyclictest87550irq/128-enp1s0-08:20:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional