You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 08:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Thu Apr 23, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
184762890,1sleep10-21swapper/123:36:341
534228767,9sleep30-21swapper/319:09:563
501228774,8sleep00-21swapper/019:05:380
518028644,38sleep20-21swapper/219:07:492
501828265,13sleep10-21swapper/119:05:431
544199309,20cyclictest31010-21diskmemload00:02:081
5441992925,3cyclictest0-21swapper/121:20:031
544399280,26cyclictest0-21swapper/221:30:322
5447992724,2cyclictest39-21ksoftirqd/323:24:423
544399270,27cyclictest0-21swapper/223:11:302
544399270,26cyclictest0-21swapper/222:05:182
544399260,26cyclictest0-21swapper/223:34:082
544399250,24cyclictest0-21swapper/223:47:282
544399250,24cyclictest0-21swapper/223:47:272
544399250,1cyclictest0-21swapper/200:08:052
5441992520,2cyclictest0-21swapper/123:30:151
544399240,23cyclictest0-21swapper/222:50:352
544399240,22cyclictest0-21swapper/223:58:512
544399240,1cyclictest0-21swapper/223:05:162
5441992415,8cyclictest0-21swapper/121:58:091
5441992415,8cyclictest0-21swapper/121:58:081
5441992414,9cyclictest31010-21diskmemload23:16:101
5441992412,11cyclictest0-21swapper/100:15:201
5441992410,13cyclictest0-21swapper/122:06:461
543799240,2cyclictest0-21swapper/022:16:400
5447992318,4cyclictest39-21ksoftirqd/300:35:443
544399230,0cyclictest0-21swapper/223:28:192
5441992321,1cyclictest0-21swapper/123:50:131
5441992320,2cyclictest32641-21ssh00:05:131
5441992312,8cyclictest0-21swapper/123:05:451
544399220,22cyclictest0-21swapper/221:43:262
544399220,22cyclictest0-21swapper/200:31:502
544199227,11cyclictest0-21swapper/120:20:131
5441992219,2cyclictest0-21swapper/121:39:001
5441992212,9cyclictest0-21swapper/122:15:191
5441992211,6cyclictest31010-21diskmemload22:52:321
5441992210,4cyclictest0-21swapper/100:20:081
544199220,12cyclictest0-21swapper/120:50:111
5447992113,2cyclictest39-21ksoftirqd/323:40:133
544399210,20cyclictest0-21swapper/200:39:192
544199214,15cyclictest0-21swapper/122:22:241
5441992119,1cyclictest0-21swapper/120:25:131
5441992111,9cyclictest31010-21diskmemload22:45:011
544199210,20cyclictest0-21swapper/122:36:041
543799216,8cyclictest16660-21munin-run00:15:010
5437992118,1cyclictest13-21ksoftirqd/022:48:490
5443992019,1cyclictest0-21swapper/221:28:032
544399200,20cyclictest0-21swapper/222:39:332
544399200,18cyclictest0-21swapper/223:03:322
5441992018,1cyclictest9307-21kworker/1:221:00:141
5441992018,1cyclictest0-21swapper/121:30:281
5441992018,1cyclictest0-21swapper/100:31:581
5441992017,1cyclictest0-21swapper/121:19:491
5441992013,6cyclictest31010-21diskmemload23:04:521
5441992010,7cyclictest0-21swapper/121:10:131
544199200,4cyclictest0-21swapper/122:55:461
543799200,6cyclictest0-21swapper/023:35:130
5447991916,2cyclictest39-21ksoftirqd/300:15:033
544399194,4cyclictest32-21ksoftirqd/221:22:232
544399190,18cyclictest0-21swapper/200:29:292
544199199,8cyclictest0-21swapper/123:13:221
544199199,4cyclictest0-21swapper/122:42:131
544199196,8cyclictest0-21swapper/123:29:171
5441991917,1cyclictest0-21swapper/122:25:141
5441991916,2cyclictest0-21swapper/123:40:151
5441991912,7cyclictest0-21swapper/123:47:321
5441991912,7cyclictest0-21swapper/123:47:311
5441991912,6cyclictest0-21swapper/100:25:441
5441991911,3cyclictest0-21swapper/119:59:581
5441991910,8cyclictest0-21swapper/123:55:341
75562180,0sleep30-21swapper/323:50:073
5447991816,2cyclictest39-21ksoftirqd/300:10:233
5447991814,4cyclictest3567-21apache221:24:003
544399180,18cyclictest0-21swapper/223:16:292
544199186,11cyclictest0-21swapper/100:11:031
544199185,10cyclictest0-21swapper/119:35:141
5441991816,1cyclictest0-21swapper/122:10:131
5441991816,1cyclictest0-21swapper/100:37:511
5441991810,6cyclictest31010-21diskmemload21:42:511
544199180,1cyclictest0-21swapper/121:46:311
5447991713,1cyclictest39-21ksoftirqd/322:42:423
5447991710,6cyclictest26023-21latency_hist21:00:003
544799170,6cyclictest8436-21ntpq20:20:203
5443991716,1cyclictest32-21ksoftirqd/221:55:132
5443991716,1cyclictest32-21ksoftirqd/221:55:122
5443991716,1cyclictest0-21swapper/221:39:022
5443991715,2cyclictest10838-21kworker/u8:023:36:592
544199174,10cyclictest0-21swapper/123:22:191
544799160,1cyclictest141rcu_preempt21:25:143
5443991612,3cyclictest31010-21diskmemload21:13:522
544199164,10cyclictest0-21swapper/122:31:301
5441991610,5cyclictest0-21swapper/121:50:581
543799160,1cyclictest0-21swapper/022:57:010
544799152,4cyclictest141rcu_preempt23:11:103
5447991514,1cyclictest39-21ksoftirqd/322:24:433
5447991512,2cyclictest27157-21cat22:05:143
544799150,14cyclictest0-21swapper/300:22:413
544399153,7cyclictest516-21gdbus22:20:012
544399150,15cyclictest0-21swapper/222:58:122
544399150,15cyclictest0-21swapper/221:19:432
5441991511,3cyclictest31010-21diskmemload21:26:371
543799159,5cyclictest0-21swapper/022:32:090
543799154,6cyclictest16549-21sendmail-msp20:40:000
5437991514,1cyclictest0-21swapper/021:00:150
543799150,0cyclictest0-21swapper/000:33:550
544799146,5cyclictest225-21systemd-journal19:54:043
544799145,2cyclictest141rcu_preempt21:11:433
5447991412,2cyclictest39-21ksoftirqd/323:35:203
5447991411,2cyclictest10991-21wc21:35:243
544799141,10cyclictest7660-21/usr/sbin/munin21:15:173
544799140,1cyclictest141rcu_preempt00:30:293
544799140,1cyclictest0-21swapper/323:18:393
5443991413,1cyclictest30776-21ssh23:44:402
544399141,2cyclictest32-21ksoftirqd/222:29:062
544199147,6cyclictest24443-21threads19:45:261
544199142,11cyclictest0-21swapper/119:40:151
5441991410,2cyclictest17783-21wc20:40:191
5441991410,1cyclictest0-21swapper/119:10:111
543799146,4cyclictest22633-21runrttasks21:42:510
543799141,9cyclictest0-21swapper/020:40:140
5437991414,0cyclictest0-21swapper/000:24:180
5437991411,2cyclictest22216-21ssh23:00:160
543799140,1cyclictest0-21swapper/000:02:230
544799132,5cyclictest225-21systemd-journal00:09:583
544799132,10cyclictest25397-21kworker/u8:023:04:283
544799131,9cyclictest14336-21latency_hist20:35:003
5447991312,1cyclictest0-21swapper/321:56:463
5447991312,1cyclictest0-21swapper/321:56:463
5447991312,1cyclictest0-21swapper/300:01:363
5447991311,2cyclictest39-21ksoftirqd/322:31:153
5447991310,3cyclictest381rcuc/321:34:573
544799130,13cyclictest0-21swapper/322:51:543
5443991311,1cyclictest32-21ksoftirqd/222:10:082
5443991311,1cyclictest32-21ksoftirqd/221:51:132
5443991310,2cyclictest9475-21cut00:10:172
5443991310,2cyclictest8274-21tr19:15:142
5443991310,2cyclictest27230-21grep21:45:202
5443991310,2cyclictest23691-21tail19:45:172
544399130,11cyclictest0-21swapper/200:20:042
544199139,2cyclictest11431-21wc19:20:181
543799139,3cyclictest31215-21grep20:00:220
543799139,3cyclictest13533-21cron23:15:000
543799138,3cyclictest0-21swapper/023:23:470
543799134,8cyclictest14543-21perf23:35:000
543799131,6cyclictest32500-21apt-get21:30:000
5437991310,2cyclictest8978-21tr23:50:200
5437991310,2cyclictest8699-21ssh21:35:010
5437991310,2cyclictest19910-21cat22:39:590
5437991310,2cyclictest11065-21grep19:20:160
5437991310,2cyclictest10251-21grep20:25:150
543799130,2cyclictest0-21swapper/022:54:530
544799129,2cyclictest7555-21sed23:30:173
544799129,2cyclictest4626-21basename20:15:003
544799129,2cyclictest31023-21basename21:10:003
544799129,2cyclictest16744-21tune2fs23:55:163
544799129,2cyclictest12789-21sort22:15:253
544799128,4cyclictest25645-21taskset22:04:343
544799128,3cyclictest7965-21cat19:15:013
544799128,3cyclictest30067-21grep23:05:143
544799128,3cyclictest14599-21cat19:25:253
544799125,3cyclictest477-21rs:main3
544799123,1cyclictest141rcu_preempt22:25:273
544799121,7cyclictest15657-21grep19:30:153
544799120,11cyclictest21457-21cat20:50:003
544399129,2cyclictest28001-21tr19:55:162
544399129,2cyclictest22628-21tail20:50:182
544399129,2cyclictest20215-21tail22:20:152
544399128,3cyclictest829-21sed20:05:182
544399128,3cyclictest829-21sed20:05:172
544399128,3cyclictest22675-21tr23:20:142
544399127,4cyclictest17052-21sed19:30:272
544399126,1cyclictest141rcu_preempt20:25:262
544399120,2cyclictest0-21swapper/222:40:592
544399120,1cyclictest32-21ksoftirqd/222:30:442
544399120,11cyclictest7564-21tr20:20:142
544399120,0cyclictest0-21swapper/223:52:252
544199129,2cyclictest13796-21head19:25:161
544199125,6cyclictest0-21swapper/120:30:111
544199125,3cyclictest225-21systemd-journal19:19:591
544199124,6cyclictest0-21swapper/120:10:161
544199123,4cyclictest24908-21sh22:03:591
5441991210,1cyclictest0-21swapper/119:30:131
543799129,2cyclictest9458-21tr21:35:130
543799129,2cyclictest9416-21cat20:25:000
543799129,2cyclictest8825-21head19:15:160
543799129,2cyclictest32568-21grep23:45:170
543799129,2cyclictest32568-21grep23:45:160
543799129,2cyclictest20233-21grep00:35:260
543799129,2cyclictest19429-21grep20:45:130
543799129,2cyclictest18242-21grep19:35:150
543799128,3cyclictest21778-21grep20:50:130
543799128,3cyclictest18541-21grep22:00:140
543799124,7cyclictest27277-21munin-run22:25:000
543799123,1cyclictest31010-21diskmemload23:26:590
543799121,1cyclictest141rcu_preempt23:06:570
543799120,12cyclictest0-21swapper/023:59:540
544799119,1cyclictest29403-21chrt22:45:153
544799119,1cyclictest0-21swapper/321:42:013
544799118,3cyclictest1412-21ssh23:45:313
544799118,3cyclictest1412-21ssh23:45:313
544799118,2cyclictest6300-21cat20:15:243
544799118,2cyclictest3075-21ssh00:25:223
544799118,2cyclictest25694-21ssh21:45:063
544799118,2cyclictest2549-21grep21:50:183
544799118,2cyclictest24199-21head19:45:223
544799118,2cyclictest2285-21sh23:26:593
544799118,2cyclictest20702-21cut19:40:153
544799118,2cyclictest19108-21cut20:44:593
544799118,2cyclictest19058-21cat19:35:193
544799118,2cyclictest11825-21ssh22:35:043
544799118,2cyclictest1143-21cat20:05:203
544799118,2cyclictest1143-21cat20:05:193
544799118,2cyclictest11327-21sed20:25:263
544399119,1cyclictest26027-21ls21:00:002
544399118,2cyclictest7175-21grep19:10:242
544399118,2cyclictest4582-21chrt20:14:482
544399118,2cyclictest31765-21ssh22:46:132
544399118,2cyclictest30819-21tr20:00:182
544399118,2cyclictest296902sleep221:05:172
544399118,2cyclictest28408-21cron21:04:592
544399118,2cyclictest25300-21ssh22:04:182
544399118,2cyclictest24960-21ssh00:00:162
544399118,2cyclictest20913-21ssh00:16:492
544399118,2cyclictest16204-21chrt20:36:162
544399118,2cyclictest14836-21taskset19:26:172
544399118,2cyclictest12796-21cut20:30:162
5443991110,1cyclictest32-21ksoftirqd/219:20:142
544399110,10cyclictest20009-21taskset19:39:012
544399110,10cyclictest17570-21cut20:40:172
544199119,1cyclictest0-21swapper/120:55:151
544199118,2cyclictest28700-21date21:05:001
544199118,2cyclictest0-21swapper/120:20:011
544199117,3cyclictest0-21swapper/120:05:141
544199117,3cyclictest0-21swapper/120:05:131
544199115,5cyclictest0-21swapper/120:35:131
544199113,4cyclictest30605-21df_abs20:00:161
543799118,2cyclictest8609-21ssh21:15:180
543799118,2cyclictest5656-21head20:15:170
543799118,2cyclictest4610-21sh20:15:000
543799118,2cyclictest3280-21grep22:10:160
543799118,2cyclictest31805-21grep21:10:150
543799118,2cyclictest29529-21ssh23:43:420
543799118,2cyclictest26894-21cat19:50:250
543799118,2cyclictest25288-21cat19:50:010
543799118,2cyclictest21535-21uname19:40:180
543799118,2cyclictest2045-21expr00:25:140
543799118,2cyclictest19066-21sed00:15:250
543799118,2cyclictest16178-21ssh23:15:320
543799118,2cyclictest15833-21expr19:30:150
543799118,2cyclictest10786-21ssh21:55:160
543799118,2cyclictest10786-21ssh21:55:150
543799117,3cyclictest28411-21ssh22:25:160
543799113,4cyclictest225-21systemd-journal00:09:570
543799111,9cyclictest0-21swapper/021:54:250
543799110,9cyclictest0-21swapper/021:21:330
544799108,1cyclictest27041-21grep21:00:173
544799108,1cyclictest128302sleep322:55:013
544799108,1cyclictest10825-21cat19:20:143
544799107,2cyclictest4884-21ssh22:10:373
544799107,2cyclictest30522-21expr20:00:163
544799107,2cyclictest21543-21apt-get20:50:113
544399108,1cyclictest209922sleep220:46:302
544399108,1cyclictest20599-21taskset19:40:122
544399104,2cyclictest5596-21grep20:15:172
544399100,1cyclictest141rcu_preempt19:50:122
544199106,3cyclictest25620-21tr19:50:141
544199103,2cyclictest0-21swapper/120:45:141
543799108,1cyclictest6517-21mii-tool19:10:160
543799108,1cyclictest29996-21sed21:05:190
543799108,1cyclictest27584-21rm22:05:170
543799108,1cyclictest26245-21expr21:45:140
543799108,1cyclictest239212sleep020:55:130
543799108,1cyclictest22738-21sh22:40:430
543799108,1cyclictest12906-21gpg-connect-age19:25:000
543799107,2cyclictest32447-21taskset20:05:030
543799107,2cyclictest32447-21taskset20:05:020
543799107,2cyclictest12721-21cut20:30:170
543799100,9cyclictest29620-21taskset19:57:300
54479998,1cyclictest27861-21grep19:55:163
54479998,1cyclictest14502-21wget20:35:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional