You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 08:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Thu Jul 09, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
83882240165,23sleep20-21swapper/219:09:512
80362222192,20sleep10-21swapper/119:05:271
81292202165,25sleep00-21swapper/019:06:270
81222201164,25sleep30-21swapper/319:06:223
3116921790,4sleep1849199cyclictest21:10:241
3116921790,4sleep1849199cyclictest21:10:241
1630921620,3sleep3850399cyclictest19:25:123
136721530,5sleep1849199cyclictest22:30:231
2837421510,5sleep2849699cyclictest23:32:202
2837421510,5sleep2849699cyclictest23:32:202
8496991091,2cyclictest0-21swapper/221:15:592
849699105101,3cyclictest0-21swapper/220:33:062
322882640,1sleep00-21swapper/021:15:130
322122510,3sleep332216-21latency20:00:193
174642480,2sleep20-21swapper/221:55:122
152362480,2sleep10-21swapper/119:20:241
39072450,2sleep10-21swapper/122:35:241
8503994426,3cyclictest0-21swapper/300:20:593
8496994340,2cyclictest0-21swapper/222:57:432
8496994127,9cyclictest0-21swapper/220:20:012
8496994026,2cyclictest0-21swapper/223:20:202
849699401,21cyclictest0-21swapper/223:01:352
849699401,13cyclictest0-21swapper/200:35:272
848699391,26cyclictest23725-21kworker/0:219:10:230
849699381,30cyclictest0-21swapper/219:55:092
8503993720,3cyclictest13786-21users20:30:273
849699371,20cyclictest0-21swapper/200:13:512
8486993710,13cyclictest21400-21perf22:05:010
849699361,25cyclictest0-21swapper/220:59:272
849699351,32cyclictest0-21swapper/219:14:252
850399341,13cyclictest6581-21switchtime23:55:233
8496993417,3cyclictest21978-21strings20:50:192
849699341,26cyclictest0-21swapper/221:04:112
849699341,23cyclictest0-21swapper/200:30:192
849699340,12cyclictest0-21swapper/222:11:092
121942340,2sleep20-21swapper/219:15:162
8503993319,3cyclictest0-21swapper/319:40:343
849699331,9cyclictest14125-21dnf19:20:142
8496993319,4cyclictest0-21swapper/220:35:552
849699331,2cyclictest0-21swapper/219:33:492
849699331,23cyclictest0-21swapper/219:39:132
849699331,20cyclictest0-21swapper/223:58:112
849699331,19cyclictest0-21swapper/221:51:432
8503993217,6cyclictest0-21swapper/319:50:293
850399321,20cyclictest0-21swapper/321:19:433
850399320,28cyclictest29558-21diskmemload21:49:113
8496993217,3cyclictest0-21swapper/222:25:192
849699321,24cyclictest0-21swapper/200:03:472
849699321,22cyclictest0-21swapper/221:10:162
849699321,22cyclictest0-21swapper/221:10:152
849699321,19cyclictest0-21swapper/223:40:152
849699321,12cyclictest0-21swapper/223:52:552
849699321,12cyclictest0-21swapper/223:49:512
849699320,21cyclictest0-21swapper/221:30:512
8486993211,8cyclictest7240-21kworker/0:120:40:150
849699311,22cyclictest0-21swapper/223:28:592
849699311,18cyclictest0-21swapper/222:43:352
849699311,17cyclictest11616-21netstat21:40:192
848699311,15cyclictest6617-21idleruntime-cro22:45:010
8503993019,3cyclictest0-21swapper/321:01:353
8503993018,3cyclictest0-21swapper/319:14:253
850399301,12cyclictest0-21swapper/319:47:533
8496993027,2cyclictest0-21swapper/219:42:172
849699301,21cyclictest0-21swapper/200:15:472
849699301,20cyclictest0-21swapper/221:27:162
849699301,19cyclictest0-21swapper/220:06:132
849699301,14cyclictest0-21swapper/223:16:522
849699301,14cyclictest0-21swapper/200:05:352
849699301,12cyclictest0-21swapper/222:05:352
848699304,12cyclictest24851-21perf00:40:020
848699301,24cyclictest19680-21kworker/0:123:14:410
23192300,6sleep081ktimersoftd/020:05:230
8503992920,3cyclictest0-21swapper/319:33:493
8503992912,8cyclictest0-21swapper/320:35:313
850399291,18cyclictest0-21swapper/321:40:273
850399290,13cyclictest0-21swapper/323:38:513
849699291,20cyclictest0-21swapper/222:03:032
849699291,13cyclictest0-21swapper/220:20:172
849699291,12cyclictest0-21swapper/221:47:112
849699291,12cyclictest0-21swapper/219:51:012
849699291,12cyclictest0-21swapper/219:29:412
849699291,11cyclictest0-21swapper/222:15:592
848699295,14cyclictest87650irq/129-enp1s0-22:10:390
850399281,10cyclictest0-21swapper/321:35:173
8496992816,3cyclictest0-21swapper/200:27:392
849699281,11cyclictest0-21swapper/223:38:152
849699281,10cyclictest0-21swapper/221:36:132
849699280,12cyclictest0-21swapper/220:25:452
848699282,18cyclictest5764-21kworker/0:121:28:410
848699282,10cyclictest12478-21kworker/0:121:50:210
850399273,8cyclictest0-21swapper/319:37:463
849699270,10cyclictest0-21swapper/223:05:232
849199279,4cyclictest827-21runrttasks19:29:071
849199271,12cyclictest0-21swapper/123:43:471
848699278,16cyclictest20458-21awk00:30:010
848699271,9cyclictest27888-21kworker/0:022:21:500
848699271,13cyclictest16050-21munin-run00:20:000
848699271,11cyclictest26481-21kworker/0:319:53:550
848699271,10cyclictest2673-21kworker/0:023:59:290
848699271,10cyclictest12478-21kworker/0:121:49:250
252502270,2sleep1221ktimersoftd/121:00:011
850399263,8cyclictest0-21swapper/320:42:553
8503992613,3cyclictest0-21swapper/320:45:073
8503992612,3cyclictest0-21swapper/322:37:393
850399261,15cyclictest9376-21df_inode22:50:143
850399261,13cyclictest0-21swapper/322:11:313
850399261,12cyclictest0-21swapper/322:21:513
849699261,19cyclictest0-21swapper/220:48:032
849699261,13cyclictest0-21swapper/222:50:552
849699261,12cyclictest0-21swapper/223:13:362
849699261,12cyclictest0-21swapper/219:46:532
849699261,11cyclictest0-21swapper/222:23:552
849699261,11cyclictest0-21swapper/220:42:312
849699261,10cyclictest0-21swapper/200:23:492
849199261,11cyclictest14789-21chrt21:50:011
848699262,13cyclictest30133-21kworker/0:122:26:500
848699262,10cyclictest8028-21kworker/0:221:34:530
313852260,4sleep3850399cyclictest23:40:173
850399258,3cyclictest0-21swapper/320:05:093
850399256,3cyclictest0-21swapper/321:21:553
850399253,8cyclictest0-21swapper/323:06:553
8503992513,7cyclictest0-21swapper/320:12:173
850399251,11cyclictest8135-21df_inode20:20:133
850399251,10cyclictest8190-21awk21:35:013
849699259,3cyclictest0-21swapper/220:13:292
849699258,3cyclictest0-21swapper/221:20:402
8496992510,3cyclictest0-21swapper/222:49:072
849699250,10cyclictest0-21swapper/221:07:092
848699257,15cyclictest17621-21basename23:10:000
848699252,11cyclictest7240-21kworker/0:120:18:240
8486992515,6cyclictest12023-21if_err_enp2s022:55:170
8486992512,3cyclictest27888-21kworker/0:022:37:530
8486992511,3cyclictest21905-21kworker/0:223:35:410
848699251,10cyclictest29372-21kworker/0:221:08:050
848699251,10cyclictest26481-21kworker/0:319:55:270
848699251,10cyclictest10415-21kworker/0:319:29:430
139372250,2sleep011-21rcuc/019:20:110
850399248,3cyclictest0-21swapper/323:11:113
850399248,3cyclictest0-21swapper/320:51:433
850399247,3cyclictest0-21swapper/321:06:113
850399246,3cyclictest0-21swapper/322:33:073
850399246,11cyclictest19127-21kworker/u8:222:40:363
850399243,11cyclictest89150irq/134-enp2s0-23:02:273
850399241,10cyclictest20672-21/usr/sbin/munin00:30:193
8503992410,2cyclictest808-21snmpd23:46:073
850399240,9cyclictest0-21swapper/300:05:373
849699241,13cyclictest0-21swapper/222:39:032
849699241,13cyclictest0-21swapper/222:33:312
849699241,10cyclictest0-21swapper/220:02:572
849199245,10cyclictest579-21systemd-journal21:40:021
849199242,16cyclictest0-21swapper/122:28:131
849199241,16cyclictest0-21swapper/100:27:371
848699242,9cyclictest8538-21kworker/0:023:16:410
848699241,9cyclictest1846-21kworker/0:222:33:330
848699241,8cyclictest7240-21kworker/0:120:47:170
848699241,5cyclictest22614-21netstat22:05:190
848699241,10cyclictest7240-21kworker/0:120:22:110
848699241,10cyclictest30464-21kworker/0:123:50:370
848699241,10cyclictest23351-21kworker/0:122:15:410
848699241,10cyclictest22652-21kworker/0:221:00:370
848699241,10cyclictest2119-21kworker/0:322:53:210
68472240,5sleep121-21rcuc/120:15:221
850399236,3cyclictest0-21swapper/321:58:363
850399234,3cyclictest0-21swapper/320:27:233
850399234,12cyclictest15209-21expr21:50:123
850399233,10cyclictest0-21swapper/323:22:473
850399231,9cyclictest0-21swapper/322:25:493
850399231,9cyclictest0-21swapper/300:01:433
850399231,8cyclictest0-21swapper/319:58:293
850399231,11cyclictest6719-21pmu-power20:15:213
850399230,10cyclictest0-21swapper/300:15:593
849199238,4cyclictest0-21swapper/123:55:511
849199236,5cyclictest26340-21df22:15:121
849199232,12cyclictest26393-21munin-run23:30:011
849199231,7cyclictest0-21swapper/121:32:431
8491992317,3cyclictest4689-21/usr/sbin/munin21:25:141
8491992315,3cyclictest0-21swapper/121:54:431
849199231,11cyclictest0-21swapper/121:41:451
848699239,3cyclictest20432-21kworker/0:100:32:370
848699239,3cyclictest10248-21kworker/0:021:40:370
848699238,4cyclictest2673-21kworker/0:000:20:370
848699232,9cyclictest8538-21kworker/0:022:49:170
848699231,9cyclictest5764-21kworker/0:121:35:490
848699231,9cyclictest11560-21kworker/0:100:11:250
848699231,10cyclictest30464-21kworker/0:123:47:410
850399228,3cyclictest4735-21if_enp1s021:25:153
850399226,3cyclictest0-21swapper/320:56:153
850399225,8cyclictest0-21swapper/322:58:513
850399224,8cyclictest87350irq/126-enp1s0-23:53:133
850399224,3cyclictest0-21swapper/323:15:593
850399223,8cyclictest0-21swapper/322:46:513
850399223,3cyclictest0-21swapper/323:27:513
850399222,11cyclictest0-21swapper/300:11:033
849199228,4cyclictest0-21swapper/122:12:301
849199227,9cyclictest0-21swapper/121:24:361
849199222,15cyclictest0-21swapper/122:05:461
8491992216,3cyclictest23866-21netstat00:35:191
849199221,18cyclictest840-21sensors21:15:241
849199221,18cyclictest16545-21load23:05:181
849199221,18cyclictest12304-21ntp_states22:55:231
849199221,11cyclictest0-21swapper/122:20:131
849199220,3cyclictest0-21swapper/123:23:181
848699227,4cyclictest7240-21kworker/0:120:33:530
848699227,3cyclictest356-21kworker/0:323:44:090
848699225,2cyclictest19821-21kworker/0:119:35:190
848699222,9cyclictest21905-21kworker/0:223:30:050
848699222,9cyclictest21905-21kworker/0:223:30:050
848699222,8cyclictest20410-21kworker/0:020:50:090
848699221,9cyclictest26481-21kworker/0:320:01:070
848699221,9cyclictest24125-21kworker/0:123:24:570
848699221,9cyclictest21905-21kworker/0:223:28:210
848699221,12cyclictest7240-21kworker/0:120:38:130
848699221,12cyclictest3611-21kworker/0:020:12:320
848699221,12cyclictest20410-21kworker/0:020:59:410
848699221,10cyclictest2673-21kworker/0:000:01:570
848699220,8cyclictest2673-21kworker/0:000:06:130
54462220,1sleep10-21swapper/122:40:181
850399215,7cyclictest0-21swapper/322:18:393
850399215,3cyclictest0-21swapper/322:00:363
850399214,3cyclictest0-21swapper/322:05:153
850399214,10cyclictest87350irq/126-enp1s0-00:27:053
850399213,9cyclictest0-21swapper/300:36:273
850399213,2cyclictest87450irq/127-enp1s0-19:15:293
849199213,6cyclictest18245-1kworker/u9:222:53:371
849199212,15cyclictest0-21swapper/122:48:341
849199211,6cyclictest0-21swapper/100:22:311
849199211,3cyclictest2088-21latency20:05:181
849199211,18cyclictest0-21swapper/120:48:021
849199211,10cyclictest0-21swapper/122:01:101
8491992110,3cyclictest18122-21memory21:55:181
849199210,16cyclictest87550irq/128-enp1s0-23:33:181
849199210,16cyclictest87550irq/128-enp1s0-23:33:181
848699218,3cyclictest1301-21kworker/0:121:20:170
848699212,10cyclictest12999-21kworker/0:123:01:450
848699211,12cyclictest22073-21kworker/0:019:49:030
850399201,8cyclictest0-21swapper/319:21:053
850399201,10cyclictest0-21swapper/321:11:073
850399201,10cyclictest0-21swapper/321:11:073
849199209,3cyclictest5020-21latency_hist23:55:011
849199202,16cyclictest0-21swapper/121:06:091
8491992016,3cyclictest0-21swapper/120:00:071
849199201,18cyclictest0-21swapper/123:19:271
849199201,17cyclictest0-21swapper/119:15:181
849199201,10cyclictest8933-21pmu-power20:20:221
849199201,10cyclictest20834-21grep00:30:121
849199201,10cyclictest10800-21munin-node19:15:021
849199201,10cyclictest0-21swapper/123:12:481
848699207,3cyclictest20410-21kworker/0:021:12:490
848699207,3cyclictest20410-21kworker/0:021:12:490
848699203,9cyclictest755-21rngd19:32:010
848699202,8cyclictest7240-21kworker/0:120:27:170
850399191,10cyclictest29558-21diskmemload23:30:163
850399191,10cyclictest29558-21diskmemload23:30:153
849199196,3cyclictest87550irq/128-enp1s0-19:40:241
849199192,5cyclictest14740-21diskstats20:35:131
849199192,16cyclictest0-21swapper/123:04:151
849199192,16cyclictest0-21swapper/120:30:431
8491991916,2cyclictest0-21swapper/120:41:271
8491991916,2cyclictest0-21swapper/120:10:341
849199191,4cyclictest0-21swapper/123:47:121
8491991914,3cyclictest29479-21cut23:35:151
849199191,3cyclictest0-21swapper/119:30:401
849199191,2cyclictest0-21swapper/120:52:191
849199191,17cyclictest0-21swapper/100:01:011
849199191,16cyclictest0-21swapper/119:35:061
849199191,11cyclictest0-21swapper/119:45:201
849199191,10cyclictest29166-21basename19:55:101
849199191,10cyclictest14981-21latency00:15:161
849199190,17cyclictest0-21swapper/121:00:171
848699191,10cyclictest24471-21sshd19:44:160
8491991815,2cyclictest0-21swapper/120:25:261
849199181,16cyclictest0-21swapper/119:51:261
849199181,16cyclictest0-21swapper/100:10:171
849199181,16cyclictest0-21swapper/100:05:081
8486991813,2cyclictest10415-21kworker/0:319:15:230
848699175,3cyclictest12478-21kworker/0:121:56:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional